This page contains automatically translated content.

M. Sc. Malte Lehna

Reinforcement Learning for Cognitive Energy Systems (RL4CES)

Email: malte.lehna[at]uni-kassel[dot]de

Publications

[ 2026 ]
[ 2024 ]
[ 2023 ]

2026[ to top ]

Learning Topology Actions for Power Grid Control: A Graph-Based Soft-Label Imitation Learning Approach.Hassouna, Mohamed; Holzhüter, Clara; Lehna, Malte; de Jong, Matthijs; Viebahn, Jan; Sick, Bernhard; Scholz, Christoph. In European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECMLPKDD), bll 129–146. Springer, 2026.
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
- [ URL ]
The rising proportion of renewable energy in the electricity mix introduces significant operational challenges for power grid operators. Effective power grid management demands adaptive decision-making strategies capable of handling dynamic conditions. With the increase in complexity, more and more Deep Learning (DL) approaches have been proposed to find suitable grid topologies for congestion management. In this work, we contribute to this research by introducing a novel Imitation Learning (IL) approach that leverages soft labels derived from simulated topological action outcomes, thereby capturing multiple viable actions per state. Unlike traditional IL methods that rely on hard labels to enforce a single optimal action, our method constructs soft labels that capture the effectiveness of actions that prove suitable in resolving grid congestion. To further enhance decision-making, we integrate Graph Neural Networks (GNNs) to encode the structural properties of power grids, ensuring that the topology-aware representations contribute to better agent performance. Our approach significantly outperforms its hard-label counterparts as well as state-of-the-art Deep Reinforcement Learning (DRL) baseline agents. Most notably, it achieves a 17% better performance compared to the greedy expert agent from which the imitation targets were derived.
@inproceedings{hassouna2025learning, abstract = {The rising proportion of renewable energy in the electricity mix introduces significant operational challenges for power grid operators. Effective power grid management demands adaptive decision-making strategies capable of handling dynamic conditions. With the increase in complexity, more and more Deep Learning (DL) approaches have been proposed to find suitable grid topologies for congestion management. In this work, we contribute to this research by introducing a novel Imitation Learning (IL) approach that leverages soft labels derived from simulated topological action outcomes, thereby capturing multiple viable actions per state. Unlike traditional IL methods that rely on hard labels to enforce a single optimal action, our method constructs soft labels that capture the effectiveness of actions that prove suitable in resolving grid congestion. To further enhance decision-making, we integrate Graph Neural Networks (GNNs) to encode the structural properties of power grids, ensuring that the topology-aware representations contribute to better agent performance. Our approach significantly outperforms its hard-label counterparts as well as state-of-the-art Deep Reinforcement Learning (DRL) baseline agents. Most notably, it achieves a 17% better performance compared to the greedy expert agent from which the imitation targets were derived.}, author = {Hassouna, Mohamed and Holzhüter, Clara and Lehna, Malte and de Jong, Matthijs and Viebahn, Jan and Sick, Bernhard and Scholz, Christoph}, booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECMLPKDD)}, keywords = {itegpub}, pages = {129–146}, publisher = {Springer}, title = {Learning Topology Actions for Power Grid Control: A Graph-Based Soft-Label Imitation Learning Approach}, year = 2026 }
%0 Conference Paper %1 hassouna2025learning %A Hassouna, Mohamed %A Holzhüter, Clara %A Lehna, Malte %A de Jong, Matthijs %A Viebahn, Jan %A Sick, Bernhard %A Scholz, Christoph %B European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECMLPKDD) %D 2026 %I Springer %P 129--146 %R 10.1007/978-3-032-06129-4_8 %T Learning Topology Actions for Power Grid Control: A Graph-Based Soft-Label Imitation Learning Approach %U https://link.springer.com/chapter/10.1007/978-3-032-06129-4_8 %X The rising proportion of renewable energy in the electricity mix introduces significant operational challenges for power grid operators. Effective power grid management demands adaptive decision-making strategies capable of handling dynamic conditions. With the increase in complexity, more and more Deep Learning (DL) approaches have been proposed to find suitable grid topologies for congestion management. In this work, we contribute to this research by introducing a novel Imitation Learning (IL) approach that leverages soft labels derived from simulated topological action outcomes, thereby capturing multiple viable actions per state. Unlike traditional IL methods that rely on hard labels to enforce a single optimal action, our method constructs soft labels that capture the effectiveness of actions that prove suitable in resolving grid congestion. To further enhance decision-making, we integrate Graph Neural Networks (GNNs) to encode the structural properties of power grids, ensuring that the topology-aware representations contribute to better agent performance. Our approach significantly outperforms its hard-label counterparts as well as state-of-the-art Deep Reinforcement Learning (DRL) baseline agents. Most notably, it achieves a 17% better performance compared to the greedy expert agent from which the imitation targets were derived.

2024[ to top ]

Fault Detection for Agents in Power Grid Topology Optimization: A Comprehensive Analysis.Lehna, Malte; Hassouna, Mohamed; Degtyar, Dmitry; Tomforde, Sven; Scholz, Christoph. In Machine Learning for Sustainable Power Systems (ML4SPS), ECML. Springer, 2024.
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
Optimizing the topology of transmission networks using Deep Reinforcement Learning (DRL) has increasingly come into focus. Various DRL agents have been proposed, which are mostly benchmarked on the Grid2Op environment from the Learning to Run a Power Network (L2RPN) challenges. The environments have many advantages with their realistic chronics and underlying power flow backends. However, the interpretation of agent survival or failure is not always clear, as there are a variety of potential causes. In this work, we focus on the failures of the power grid simulation to identify patterns and detect them in advance. We collect the failed chronics of three different agents on the WCCI 2022 L2RPN environment, totaling about 40k data points. By clustering, we are able to detect five distinct clusters, identifying common failure types. Further, we propose a multi-class prediction approach to detect failures beforehand and evaluate five different prediction models. Here, the Light Gradient-Boosting Machine (LightGBM) shows the best failure detection performance, with an accuracy of 86%. It also correctly identifies in 91% of the time failure and survival observations. Finally, we provide a detailed feature importance analysis that identifies critical features and regions in the grid.
@inproceedings{lehna2024fault, abstract = {Optimizing the topology of transmission networks using Deep Reinforcement Learning (DRL) has increasingly come into focus. Various DRL agents have been proposed, which are mostly benchmarked on the Grid2Op environment from the Learning to Run a Power Network (L2RPN) challenges. The environments have many advantages with their realistic chronics and underlying power flow backends. However, the interpretation of agent survival or failure is not always clear, as there are a variety of potential causes. In this work, we focus on the failures of the power grid simulation to identify patterns and detect them in advance. We collect the failed chronics of three different agents on the WCCI 2022 L2RPN environment, totaling about 40k data points. By clustering, we are able to detect five distinct clusters, identifying common failure types. Further, we propose a multi-class prediction approach to detect failures beforehand and evaluate five different prediction models. Here, the Light Gradient-Boosting Machine (LightGBM) shows the best failure detection performance, with an accuracy of 86%. It also correctly identifies in 91% of the time failure and survival observations. Finally, we provide a detailed feature importance analysis that identifies critical features and regions in the grid.}, author = {Lehna, Malte and Hassouna, Mohamed and Degtyar, Dmitry and Tomforde, Sven and Scholz, Christoph}, booktitle = {Machine Learning for Sustainable Power Systems (ML4SPS), ECML}, keywords = {itegpub}, note = {(accepted)}, publisher = {Springer}, title = {Fault Detection for Agents in Power Grid Topology Optimization: A Comprehensive Analysis}, year = 2024 }
%0 Conference Paper %1 lehna2024fault %A Lehna, Malte %A Hassouna, Mohamed %A Degtyar, Dmitry %A Tomforde, Sven %A Scholz, Christoph %B Machine Learning for Sustainable Power Systems (ML4SPS), ECML %D 2024 %I Springer %T Fault Detection for Agents in Power Grid Topology Optimization: A Comprehensive Analysis %X Optimizing the topology of transmission networks using Deep Reinforcement Learning (DRL) has increasingly come into focus. Various DRL agents have been proposed, which are mostly benchmarked on the Grid2Op environment from the Learning to Run a Power Network (L2RPN) challenges. The environments have many advantages with their realistic chronics and underlying power flow backends. However, the interpretation of agent survival or failure is not always clear, as there are a variety of potential causes. In this work, we focus on the failures of the power grid simulation to identify patterns and detect them in advance. We collect the failed chronics of three different agents on the WCCI 2022 L2RPN environment, totaling about 40k data points. By clustering, we are able to detect five distinct clusters, identifying common failure types. Further, we propose a multi-class prediction approach to detect failures beforehand and evaluate five different prediction models. Here, the Light Gradient-Boosting Machine (LightGBM) shows the best failure detection performance, with an accuracy of 86%. It also correctly identifies in 91% of the time failure and survival observations. Finally, we provide a detailed feature importance analysis that identifies critical features and regions in the grid.

2023[ to top ]

Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents.Lehna, Malte; Viebahn, Jan; Marot, Antoine; Tomforde, Sven; Scholz, Christoph. In Energy and AI, 14, bl 100276. Energy and AI, 2023.
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
- [ URL ]
The operation of electricity grids has become increasingly complex due to the current upheaval and the increase in renewable energy production. As a consequence, active grid management is reaching its limits with conventional approaches. In the context of the Learning to Run a Power Network (L2RPN) challenge, it has been shown that Reinforcement Learning (RL) is an efficient and reliable approach with considerable potential for automatic grid operation. In this article, we analyse the submitted agent from Binbinchen and provide novel strategies to improve the agent, both for the RL and the rule-based approach. The main improvement is a N-1 strategy, where we consider topology actions that keep the grid stable, even if one line is disconnected. More, we also propose a topology reversion to the original grid, which proved to be beneficial. The improvements are tested against reference approaches on the challenge test sets and are able to increase the performance of the rule-based agent by 27%. In direct comparison between rule-based and RL agent we find similar performance. However, the RL agent has a clear computational advantage. We also analyse the behaviour in an exemplary case in more detail to provide additional insights. Here, we observe that through the N-1 strategy, the actions of both the rule-based and the RL agent become more diversified.
@article{lehna2023managing, abstract = {The operation of electricity grids has become increasingly complex due to the current upheaval and the increase in renewable energy production. As a consequence, active grid management is reaching its limits with conventional approaches. In the context of the Learning to Run a Power Network (L2RPN) challenge, it has been shown that Reinforcement Learning (RL) is an efficient and reliable approach with considerable potential for automatic grid operation. In this article, we analyse the submitted agent from Binbinchen and provide novel strategies to improve the agent, both for the RL and the rule-based approach. The main improvement is a N-1 strategy, where we consider topology actions that keep the grid stable, even if one line is disconnected. More, we also propose a topology reversion to the original grid, which proved to be beneficial. The improvements are tested against reference approaches on the challenge test sets and are able to increase the performance of the rule-based agent by 27%. In direct comparison between rule-based and RL agent we find similar performance. However, the RL agent has a clear computational advantage. We also analyse the behaviour in an exemplary case in more detail to provide additional insights. Here, we observe that through the N-1 strategy, the actions of both the rule-based and the RL agent become more diversified.}, author = {Lehna, Malte and Viebahn, Jan and Marot, Antoine and Tomforde, Sven and Scholz, Christoph}, journal = {Energy and AI}, keywords = {itegpub}, pages = 100276, publisher = {Energy and AI}, title = {Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents}, volume = 14, year = 2023 }
%0 Journal Article %1 lehna2023managing %A Lehna, Malte %A Viebahn, Jan %A Marot, Antoine %A Tomforde, Sven %A Scholz, Christoph %D 2023 %I Energy and AI %J Energy and AI %P 100276 %R 10.1016/j.egyai.2023.100276 %T Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents %U https://www.sciencedirect.com/science/article/pii/S2666546823000484?via%3Dihub %V 14 %X The operation of electricity grids has become increasingly complex due to the current upheaval and the increase in renewable energy production. As a consequence, active grid management is reaching its limits with conventional approaches. In the context of the Learning to Run a Power Network (L2RPN) challenge, it has been shown that Reinforcement Learning (RL) is an efficient and reliable approach with considerable potential for automatic grid operation. In this article, we analyse the submitted agent from Binbinchen and provide novel strategies to improve the agent, both for the RL and the rule-based approach. The main improvement is a N-1 strategy, where we consider topology actions that keep the grid stable, even if one line is disconnected. More, we also propose a topology reversion to the original grid, which proved to be beneficial. The improvements are tested against reference approaches on the challenge test sets and are able to increase the performance of the rule-based agent by 27%. In direct comparison between rule-based and RL agent we find similar performance. However, the RL agent has a clear computational advantage. We also analyse the behaviour in an exemplary case in more detail to provide additional insights. Here, we observe that through the N-1 strategy, the actions of both the rule-based and the RL agent become more diversified.
Targeted Adversarial Attacks on Wind Power Forecasts.Heinrich, René; Scholz, Christoph; Vogt, Stephan; Lehna, Malte. In Machine Learning, 113(2), bll 863–889. Springer, 2023.
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
In recent years, researchers proposed a variety of deep learning models for wind power forecasting. These models predict the wind power generation of wind farms or entire regions more accurately than traditional machine learning algorithms or physical models. However, latest research has shown that deep learning models can often be manipulated by adversarial attacks. Since wind power forecasts are essential for the stability of modern power systems, it is important to protect them from this threat. In this work, we investigate the vulnerability of two different forecasting models to targeted, semi-targeted, and untargeted adversarial attacks. We consider a Long Short-Term Memory (LSTM) network for predicting the power generation of individual wind farms and a Convolutional Neural Network (CNN) for forecasting the wind power generation throughout Germany. Moreover, we propose the Total Adversarial Robustness Score (TARS), an evaluation metric for quantifying the robustness of regression models to targeted and semi-targeted adversarial attacks. It assesses the impact of attacks on the model's performance, as well as the extent to which the attacker's goal was achieved, by assigning a score between 0 (very vulnerable) and 1 (very robust). In our experiments, the LSTM forecasting model was fairly robust and achieved a TARS value of over 0.78 for all adversarial attacks investigated. The CNN forecasting model only achieved TARS values below 0.10 when trained ordinarily, and was thus very vulnerable. Yet, its robustness could be significantly improved by adversarial training, which always resulted in a TARS above 0.46.
@article{heinrich2023targeted, abstract = {In recent years, researchers proposed a variety of deep learning models for wind power forecasting. These models predict the wind power generation of wind farms or entire regions more accurately than traditional machine learning algorithms or physical models. However, latest research has shown that deep learning models can often be manipulated by adversarial attacks. Since wind power forecasts are essential for the stability of modern power systems, it is important to protect them from this threat. In this work, we investigate the vulnerability of two different forecasting models to targeted, semi-targeted, and untargeted adversarial attacks. We consider a Long Short-Term Memory (LSTM) network for predicting the power generation of individual wind farms and a Convolutional Neural Network (CNN) for forecasting the wind power generation throughout Germany. Moreover, we propose the Total Adversarial Robustness Score (TARS), an evaluation metric for quantifying the robustness of regression models to targeted and semi-targeted adversarial attacks. It assesses the impact of attacks on the model's performance, as well as the extent to which the attacker's goal was achieved, by assigning a score between 0 (very vulnerable) and 1 (very robust). In our experiments, the LSTM forecasting model was fairly robust and achieved a TARS value of over 0.78 for all adversarial attacks investigated. The CNN forecasting model only achieved TARS values below 0.10 when trained ordinarily, and was thus very vulnerable. Yet, its robustness could be significantly improved by adversarial training, which always resulted in a TARS above 0.46.}, author = {Heinrich, René and Scholz, Christoph and Vogt, Stephan and Lehna, Malte}, journal = {Machine Learning}, keywords = {itegpub}, number = 2, pages = {863–889}, publisher = {Springer}, title = {Targeted Adversarial Attacks on Wind Power Forecasts}, volume = 113, year = 2023 }
%0 Journal Article %1 heinrich2023targeted %A Heinrich, René %A Scholz, Christoph %A Vogt, Stephan %A Lehna, Malte %D 2023 %I Springer %J Machine Learning %N 2 %P 863--889 %R 10.1007/s10994-023-06396-9 %T Targeted Adversarial Attacks on Wind Power Forecasts %V 113 %X In recent years, researchers proposed a variety of deep learning models for wind power forecasting. These models predict the wind power generation of wind farms or entire regions more accurately than traditional machine learning algorithms or physical models. However, latest research has shown that deep learning models can often be manipulated by adversarial attacks. Since wind power forecasts are essential for the stability of modern power systems, it is important to protect them from this threat. In this work, we investigate the vulnerability of two different forecasting models to targeted, semi-targeted, and untargeted adversarial attacks. We consider a Long Short-Term Memory (LSTM) network for predicting the power generation of individual wind farms and a Convolutional Neural Network (CNN) for forecasting the wind power generation throughout Germany. Moreover, we propose the Total Adversarial Robustness Score (TARS), an evaluation metric for quantifying the robustness of regression models to targeted and semi-targeted adversarial attacks. It assesses the impact of attacks on the model's performance, as well as the extent to which the attacker's goal was achieved, by assigning a score between 0 (very vulnerable) and 1 (very robust). In our experiments, the LSTM forecasting model was fairly robust and achieved a TARS value of over 0.78 for all adversarial attacks investigated. The CNN forecasting model only achieved TARS values below 0.10 when trained ordinarily, and was thus very vulnerable. Yet, its robustness could be significantly improved by adversarial training, which always resulted in a TARS above 0.46.