M. Sc. Lukas Rauch

AI for Computationally Intelligent Systems (AI4CIS)

Fax: +49 561 804-6022
E-Mail: lukas.rauch[at]uni-kassel[dot]de

Publikationen

[ 2023 ]
[ 2022 ]

2023[ to top ]

Towards Enhancing Deep Active Learning with Weak Supervision and Constrained Clustering. Aßenmacher, Matthias; Rauch, Lukas; Goschenhofer, Jann; Stephan, Andreas; Bischl, Bernd; Roth, Benjamin; Sick, Bernhard. In Workshop on Interactive Adapative Learning (IAL), ECML PKDD, bll 65–73. 2023.
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
- [ URL ]
Three fields revolving around the question of how to cope with limited amounts of labeled data are Deep Active Learning (DAL), deep Constrained Clustering (CC), and Weakly Supervised Learning (WSL). DAL tackles the problem by adaptively posing the question of which data samples to annotate next in order to achieve the best incremental learning improvement, although it suffers from several limitations that hinder its deployment in practical settings. We point out how CC algorithms and WSL could be employed to overcome these limitations and increase the practical applicability of DAL research. Specifically, we discuss the opportunities to use the class discovery capabilities of CC and the possibility of further reducing human annotation efforts by utilizing WSL. We argue that the practical applicability of DAL algorithms will benefit from employing CC and WSL methods for the learning and labeling process. We inspect the overlaps between the three research areas and identify relevant and exciting research questions at the intersection of these areas.
@inproceedings{assenmacher2023towards, abstract = {Three fields revolving around the question of how to cope with limited amounts of labeled data are Deep Active Learning (DAL), deep Constrained Clustering (CC), and Weakly Supervised Learning (WSL). DAL tackles the problem by adaptively posing the question of which data samples to annotate next in order to achieve the best incremental learning improvement, although it suffers from several limitations that hinder its deployment in practical settings. We point out how CC algorithms and WSL could be employed to overcome these limitations and increase the practical applicability of DAL research. Specifically, we discuss the opportunities to use the class discovery capabilities of CC and the possibility of further reducing human annotation efforts by utilizing WSL. We argue that the practical applicability of DAL algorithms will benefit from employing CC and WSL methods for the learning and labeling process. We inspect the overlaps between the three research areas and identify relevant and exciting research questions at the intersection of these areas.}, author = {Aßenmacher, Matthias and Rauch, Lukas and Goschenhofer, Jann and Stephan, Andreas and Bischl, Bernd and Roth, Benjamin and Sick, Bernhard}, booktitle = {Workshop on Interactive Adapative Learning (IAL), ECML PKDD}, keywords = {itegpub}, pages = {65–73}, title = {Towards Enhancing Deep Active Learning with Weak Supervision and Constrained Clustering}, year = 2023 }
%0 Conference Paper %1 assenmacher2023towards %A Aßenmacher, Matthias %A Rauch, Lukas %A Goschenhofer, Jann %A Stephan, Andreas %A Bischl, Bernd %A Roth, Benjamin %A Sick, Bernhard %B Workshop on Interactive Adapative Learning (IAL), ECML PKDD %D 2023 %P 65--73 %T Towards Enhancing Deep Active Learning with Weak Supervision and Constrained Clustering %U https://ceur-ws.org/Vol-3470/paper7.pdf %X Three fields revolving around the question of how to cope with limited amounts of labeled data are Deep Active Learning (DAL), deep Constrained Clustering (CC), and Weakly Supervised Learning (WSL). DAL tackles the problem by adaptively posing the question of which data samples to annotate next in order to achieve the best incremental learning improvement, although it suffers from several limitations that hinder its deployment in practical settings. We point out how CC algorithms and WSL could be employed to overcome these limitations and increase the practical applicability of DAL research. Specifically, we discuss the opportunities to use the class discovery capabilities of CC and the possibility of further reducing human annotation efforts by utilizing WSL. We argue that the practical applicability of DAL algorithms will benefit from employing CC and WSL methods for the learning and labeling process. We inspect the overlaps between the three research areas and identify relevant and exciting research questions at the intersection of these areas.
DADO – Low-Cost Query Strategies for Deep Active Design Optimization. Decke, Jens; Gruhl, Christian; Rauch, Lukas; Sick, Bernhard. In International Conference on Machine Learning and Applications (ICMLA), bll 1611–1618. IEEE, 2023.
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
In this work, we apply deep active learning to the field of design optimization to reduce the number of computationally expensive numerical simulations widely used in industry and engineering. We are interested in optimizing the design of structural components, where a set of parameters describes the shape. If we can predict the performance based on these parameters and consider only the promising candidates for simulation, there is an enormous potential for saving computing power. We present two query strategies for self-optimization to reduce the computational cost in multi-objective design optimization problems. Our proposed methodology provides an intuitive approach that is easy to apply, offers significant improvements over random sampling, and circumvents the need for uncertainty estimation. We evaluate our strategies on a large dataset from the domain of fluid dynamics and introduce two new evaluation metrics to determine the model's performance. Findings from our evaluation highlights the effectiveness of our query strategies in accelerating design optimization. Furthermore, the introduced method is easily transferable to other self-optimization problems in industry and engineering.
@inproceedings{decke2023dado, abstract = {In this work, we apply deep active learning to the field of design optimization to reduce the number of computationally expensive numerical simulations widely used in industry and engineering. We are interested in optimizing the design of structural components, where a set of parameters describes the shape. If we can predict the performance based on these parameters and consider only the promising candidates for simulation, there is an enormous potential for saving computing power. We present two query strategies for self-optimization to reduce the computational cost in multi-objective design optimization problems. Our proposed methodology provides an intuitive approach that is easy to apply, offers significant improvements over random sampling, and circumvents the need for uncertainty estimation. We evaluate our strategies on a large dataset from the domain of fluid dynamics and introduce two new evaluation metrics to determine the model's performance. Findings from our evaluation highlights the effectiveness of our query strategies in accelerating design optimization. Furthermore, the introduced method is easily transferable to other self-optimization problems in industry and engineering.}, author = {Decke, Jens and Gruhl, Christian and Rauch, Lukas and Sick, Bernhard}, booktitle = {International Conference on Machine Learning and Applications (ICMLA)}, keywords = {itegpub}, pages = {1611–1618}, publisher = {IEEE}, title = {DADO – Low-Cost Query Strategies for Deep Active Design Optimization}, year = 2023 }
%0 Conference Paper %1 decke2023dado %A Decke, Jens %A Gruhl, Christian %A Rauch, Lukas %A Sick, Bernhard %B International Conference on Machine Learning and Applications (ICMLA) %D 2023 %I IEEE %P 1611--1618 %R 10.1109/ICMLA58977.2023.00244 %T DADO – Low-Cost Query Strategies for Deep Active Design Optimization %X In this work, we apply deep active learning to the field of design optimization to reduce the number of computationally expensive numerical simulations widely used in industry and engineering. We are interested in optimizing the design of structural components, where a set of parameters describes the shape. If we can predict the performance based on these parameters and consider only the promising candidates for simulation, there is an enormous potential for saving computing power. We present two query strategies for self-optimization to reduce the computational cost in multi-objective design optimization problems. Our proposed methodology provides an intuitive approach that is easy to apply, offers significant improvements over random sampling, and circumvents the need for uncertainty estimation. We evaluate our strategies on a large dataset from the domain of fluid dynamics and introduce two new evaluation metrics to determine the model's performance. Findings from our evaluation highlights the effectiveness of our query strategies in accelerating design optimization. Furthermore, the introduced method is easily transferable to other self-optimization problems in industry and engineering.
ActiveGLAE: A Benchmark for Deep Active Learning with Transformers. Rauch, Lukas; Aßenmacher, Matthias; Huseljic, Denis; Wirth, Moritz; Bischl, Bernd; Sick, Bernhard. In European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECMLPKDD), bll 55–74. Springer, 2023.
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
- [ URL ]
Deep active learning (DAL) seeks to reduce annotation costs by enabling the model to actively query instance annotations from which it expects to learn the most. Despite extensive research, there is cur- rently no standardized evaluation protocol for transformer-based lan- guage models in the field of DAL. Diverse experimental settings lead to difficulties in comparing research and deriving recommendations for prac- titioners. To tackle this challenge, we propose the ActiveGLAE bench- mark, a comprehensive collection of data sets and evaluation guidelines for assessing DAL. Our benchmark aims to facilitate and streamline the evaluation process of novel DAL strategies. Additionally, we provide an extensive overview of current practice in DAL with transformer-based language models. We identify three key challenges - data set selection, model training, and DAL settings - that pose difficulties in comparing query strategies. We establish baseline results through an extensive set of experiments as a reference point for evaluating future work. Based on our findings, we provide guidelines for researchers and practitioners.
@inproceedings{rauch2023activeglae, abstract = {Deep active learning (DAL) seeks to reduce annotation costs by enabling the model to actively query instance annotations from which it expects to learn the most. Despite extensive research, there is cur- rently no standardized evaluation protocol for transformer-based lan- guage models in the field of DAL. Diverse experimental settings lead to difficulties in comparing research and deriving recommendations for prac- titioners. To tackle this challenge, we propose the ActiveGLAE bench- mark, a comprehensive collection of data sets and evaluation guidelines for assessing DAL. Our benchmark aims to facilitate and streamline the evaluation process of novel DAL strategies. Additionally, we provide an extensive overview of current practice in DAL with transformer-based language models. We identify three key challenges - data set selection, model training, and DAL settings - that pose difficulties in comparing query strategies. We establish baseline results through an extensive set of experiments as a reference point for evaluating future work. Based on our findings, we provide guidelines for researchers and practitioners.}, author = {Rauch, Lukas and Aßenmacher, Matthias and Huseljic, Denis and Wirth, Moritz and Bischl, Bernd and Sick, Bernhard}, booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECMLPKDD)}, keywords = {itegpub}, pages = {55–74}, publisher = {Springer}, title = {ActiveGLAE: A Benchmark for Deep Active Learning with Transformers}, year = 2023 }
%0 Conference Paper %1 rauch2023activeglae %A Rauch, Lukas %A Aßenmacher, Matthias %A Huseljic, Denis %A Wirth, Moritz %A Bischl, Bernd %A Sick, Bernhard %B European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECMLPKDD) %D 2023 %I Springer %P 55--74 %R 10.1007/978-3-031-43412-9_4 %T ActiveGLAE: A Benchmark for Deep Active Learning with Transformers %U https://link.springer.com/chapter/10.1007/978-3-031-43412-9_4 %X Deep active learning (DAL) seeks to reduce annotation costs by enabling the model to actively query instance annotations from which it expects to learn the most. Despite extensive research, there is cur- rently no standardized evaluation protocol for transformer-based lan- guage models in the field of DAL. Diverse experimental settings lead to difficulties in comparing research and deriving recommendations for prac- titioners. To tackle this challenge, we propose the ActiveGLAE bench- mark, a comprehensive collection of data sets and evaluation guidelines for assessing DAL. Our benchmark aims to facilitate and streamline the evaluation process of novel DAL strategies. Additionally, we provide an extensive overview of current practice in DAL with transformer-based language models. We identify three key challenges - data set selection, model training, and DAL settings - that pose difficulties in comparing query strategies. We establish baseline results through an extensive set of experiments as a reference point for evaluating future work. Based on our findings, we provide guidelines for researchers and practitioners.
Active Bird2Vec: Towards End-To-End Bird Sound Monitoring with Transformers. Rauch, Lukas; Schwinger, Raphael; Wirth, Moritz; Sick, Bernhard; Tomforde, Sven; Scholz, Christoph. In Workshop on Artificial Intelligence for Sustainability (AI4S), ECAI, bll 1–6. 2023.
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
- [ URL ]
We propose a shift towards end-to-end learning in bird sound monitoring by combining self-supervised (SSL) and deep active learning (DAL). Leveraging transformer models, we aim to bypass traditional spectrogram conversions, enabling direct raw audio processing. ACTIVE BIRD2VEC is set to generate high-quality bird sound representations through SSL, potentially accelerating the assessment of environmental changes and decision-making processes for wind farms. Additionally, we seek to utilize the wide variety of bird vocalizations through DAL, reducing the reliance on extensively labeled datasets by human experts. We plan to curate a comprehensive set of tasks through Huggingface Datasets, enhancing future comparability and reproducibility of bioacoustic research. A comparative analysis between various transformer models will be conducted to evaluate their proficiency in bird sound recognition tasks. We aim to accelerate the progression of avian bioacoustic research and contribute to more effective conservation strategies.
@inproceedings{rauch2023active, abstract = {We propose a shift towards end-to-end learning in bird sound monitoring by combining self-supervised (SSL) and deep active learning (DAL). Leveraging transformer models, we aim to bypass traditional spectrogram conversions, enabling direct raw audio processing. ACTIVE BIRD2VEC is set to generate high-quality bird sound representations through SSL, potentially accelerating the assessment of environmental changes and decision-making processes for wind farms. Additionally, we seek to utilize the wide variety of bird vocalizations through DAL, reducing the reliance on extensively labeled datasets by human experts. We plan to curate a comprehensive set of tasks through Huggingface Datasets, enhancing future comparability and reproducibility of bioacoustic research. A comparative analysis between various transformer models will be conducted to evaluate their proficiency in bird sound recognition tasks. We aim to accelerate the progression of avian bioacoustic research and contribute to more effective conservation strategies.}, author = {Rauch, Lukas and Schwinger, Raphael and Wirth, Moritz and Sick, Bernhard and Tomforde, Sven and Scholz, Christoph}, booktitle = {Workshop on Artificial Intelligence for Sustainability (AI4S), ECAI}, keywords = {itegpub}, pages = {1–6}, title = {Active Bird2Vec: Towards End-To-End Bird Sound Monitoring with Transformers}, year = 2023 }
%0 Conference Paper %1 rauch2023active %A Rauch, Lukas %A Schwinger, Raphael %A Wirth, Moritz %A Sick, Bernhard %A Tomforde, Sven %A Scholz, Christoph %B Workshop on Artificial Intelligence for Sustainability (AI4S), ECAI %D 2023 %P 1--6 %T Active Bird2Vec: Towards End-To-End Bird Sound Monitoring with Transformers %U https://arxiv.org/abs/2308.07121 %X We propose a shift towards end-to-end learning in bird sound monitoring by combining self-supervised (SSL) and deep active learning (DAL). Leveraging transformer models, we aim to bypass traditional spectrogram conversions, enabling direct raw audio processing. ACTIVE BIRD2VEC is set to generate high-quality bird sound representations through SSL, potentially accelerating the assessment of environmental changes and decision-making processes for wind farms. Additionally, we seek to utilize the wide variety of bird vocalizations through DAL, reducing the reliance on extensively labeled datasets by human experts. We plan to curate a comprehensive set of tasks through Huggingface Datasets, enhancing future comparability and reproducibility of bioacoustic research. A comparative analysis between various transformer models will be conducted to evaluate their proficiency in bird sound recognition tasks. We aim to accelerate the progression of avian bioacoustic research and contribute to more effective conservation strategies.

2022[ to top ]

Predicting flow stress behavior of an AA7075 alloy using machine learning methods. Decke, Jens; Engelhardt, Anna; Rauch, Lukas; Degener, Sebastian; Sajjadifar, Seyedvahid; Scharifi, Emad; Steinhoff, Kurt; Niendorf, Thomas; Sick, Bernhard. In Crystals, 9(12), bll 1–19. MDPI, 2022.
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
- [ URL ]
This work focuses on the prediction of hot deformation behavior of thermo-mechanically processed precipitation hardenable aluminum alloy AA7075. Data available are focusing on a novel hot forming process at different tool temperatures ranging from 24°C to 350°C to set different cooling rates after solution-heat-treatment. Isothermal uniaxial tensile tests in the temperature range from 200°C to 400°C and at strain rates ranging from 0.001 s^-1 to 0.1 s^-1 were carried out on four different material conditions. The present paper mainly focuses on a comparative study of modeling techniques based on Machine Learning (ML) and the well-known Zerilli-Armstrong model (Z-A) as an empirically based reference. Work focused on predicting single data points of curves that the model was trained on. Due to the novel way data were split with respect to training and testing data, it becomes possible to predict entire stress-strain curves which leads to a reduction in the number of required laboratory experiments, finally saving costs and time in future experiments. While all investigated ML methods showed a higher performance than the Z-A model, the extreme Gradient Boosting model (XGB) showed the superior results, i.e., highest error reduction of 91% with respect to the Mean Squared Error.
@article{decke2022predicting, abstract = {This work focuses on the prediction of hot deformation behavior of thermo-mechanically processed precipitation hardenable aluminum alloy AA7075. Data available are focusing on a novel hot forming process at different tool temperatures ranging from 24°C to 350°C to set different cooling rates after solution-heat-treatment. Isothermal uniaxial tensile tests in the temperature range from 200°C to 400°C and at strain rates ranging from 0.001 s^{-1} to 0.1 s^{-1} were carried out on four different material conditions. The present paper mainly focuses on a comparative study of modeling techniques based on Machine Learning (ML) and the well-known Zerilli-Armstrong model (Z-A) as an empirically based reference. Work focused on predicting single data points of curves that the model was trained on. Due to the novel way data were split with respect to training and testing data, it becomes possible to predict entire stress-strain curves which leads to a reduction in the number of required laboratory experiments, finally saving costs and time in future experiments. While all investigated ML methods showed a higher performance than the Z-A model, the extreme Gradient Boosting model (XGB) showed the superior results, i.e., highest error reduction of 91% with respect to the Mean Squared Error.}, author = {Decke, Jens and Engelhardt, Anna and Rauch, Lukas and Degener, Sebastian and Sajjadifar, Seyedvahid and Scharifi, Emad and Steinhoff, Kurt and Niendorf, Thomas and Sick, Bernhard}, journal = {Crystals}, keywords = {itegpub}, number = 12, pages = {1–19}, publisher = {MDPI}, title = {Predicting flow stress behavior of an AA7075 alloy using machine learning methods}, volume = 9, year = 2022 }
%0 Journal Article %1 decke2022predicting %A Decke, Jens %A Engelhardt, Anna %A Rauch, Lukas %A Degener, Sebastian %A Sajjadifar, Seyedvahid %A Scharifi, Emad %A Steinhoff, Kurt %A Niendorf, Thomas %A Sick, Bernhard %D 2022 %I MDPI %J Crystals %N 12 %P 1--19 %R 10.3390/cryst12091281 %T Predicting flow stress behavior of an AA7075 alloy using machine learning methods %U https://www.mdpi.com/2073-4352/12/9/1281 %V 9 %X This work focuses on the prediction of hot deformation behavior of thermo-mechanically processed precipitation hardenable aluminum alloy AA7075. Data available are focusing on a novel hot forming process at different tool temperatures ranging from 24°C to 350°C to set different cooling rates after solution-heat-treatment. Isothermal uniaxial tensile tests in the temperature range from 200°C to 400°C and at strain rates ranging from 0.001 s^-1 to 0.1 s^-1 were carried out on four different material conditions. The present paper mainly focuses on a comparative study of modeling techniques based on Machine Learning (ML) and the well-known Zerilli-Armstrong model (Z-A) as an empirically based reference. Work focused on predicting single data points of curves that the model was trained on. Due to the novel way data were split with respect to training and testing data, it becomes possible to predict entire stress-strain curves which leads to a reduction in the number of required laboratory experiments, finally saving costs and time in future experiments. While all investigated ML methods showed a higher performance than the Z-A model, the extreme Gradient Boosting model (XGB) showed the superior results, i.e., highest error reduction of 91% with respect to the Mean Squared Error.
Enhancing Active Learning with Weak Supervision and Transfer Learning by Leveraging Information and Knowledge Sources. Rauch, Lukas; Huseljic, Denis; Sick, Bernhard. In Workshop on Interactive Adaptive Learning (IAL), ECML PKDD, bll 27–42. 2022.
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
- [ URL ]
One of the major limitations of deploying a machine learning model is the availability of labeled training data and the resulting expensive annotation process. Although active learning (AL) methods may reduce the annotation cost by actively selecting the most-useful instances, a costly human annotator usually provides the labels. Therefore, even with AL, we still consider the annotation process to be time-consuming and expensive. Besides human annotators, though, companies often have a vast amount of information and knowledge sources available that can generate low-cost labels (e.g., a black-box model) or improve the learning process (e.g., a pre-trained model). We present a novel approach that enhances AL with weak supervision (WS) and transfer learning (TL) to reduce the annotation cost by leveraging these sources. Specifically, we consider a black-box model like a rule-based system as an error-prone and weakly-supervised annotator that inexpensively provides labels. We estimate its performance with an annotator model to decide whether a human annotation is required. Additionally, we utilize unlabeled internal and external data by transferring knowledge from a pre-trained model to the AL cycle. We sequentially investigate the impact of WS and TL on annotation cost and model performance in an AL cycle through a use case. Our evaluation shows that our approach can reduce annotation cost by 51% while achieving nearly identical model performance compared to a traditional AL approach.
@inproceedings{rauch2022enhancing, abstract = {One of the major limitations of deploying a machine learning model is the availability of labeled training data and the resulting expensive annotation process. Although active learning (AL) methods may reduce the annotation cost by actively selecting the most-useful instances, a costly human annotator usually provides the labels. Therefore, even with AL, we still consider the annotation process to be time-consuming and expensive. Besides human annotators, though, companies often have a vast amount of information and knowledge sources available that can generate low-cost labels (e.g., a black-box model) or improve the learning process (e.g., a pre-trained model). We present a novel approach that enhances AL with weak supervision (WS) and transfer learning (TL) to reduce the annotation cost by leveraging these sources. Specifically, we consider a black-box model like a rule-based system as an error-prone and weakly-supervised annotator that inexpensively provides labels. We estimate its performance with an annotator model to decide whether a human annotation is required. Additionally, we utilize unlabeled internal and external data by transferring knowledge from a pre-trained model to the AL cycle. We sequentially investigate the impact of WS and TL on annotation cost and model performance in an AL cycle through a use case. Our evaluation shows that our approach can reduce annotation cost by 51% while achieving nearly identical model performance compared to a traditional AL approach.}, author = {Rauch, Lukas and Huseljic, Denis and Sick, Bernhard}, booktitle = {Workshop on Interactive Adaptive Learning (IAL), ECML PKDD}, keywords = {itegpub}, pages = {27–42}, title = {Enhancing Active Learning with Weak Supervision and Transfer Learning by Leveraging Information and Knowledge Sources}, year = 2022 }
%0 Conference Paper %1 rauch2022enhancing %A Rauch, Lukas %A Huseljic, Denis %A Sick, Bernhard %B Workshop on Interactive Adaptive Learning (IAL), ECML PKDD %D 2022 %P 27--42 %T Enhancing Active Learning with Weak Supervision and Transfer Learning by Leveraging Information and Knowledge Sources %U http://ceur-ws.org/Vol-3259/ialatecml_paper3.pdf %X One of the major limitations of deploying a machine learning model is the availability of labeled training data and the resulting expensive annotation process. Although active learning (AL) methods may reduce the annotation cost by actively selecting the most-useful instances, a costly human annotator usually provides the labels. Therefore, even with AL, we still consider the annotation process to be time-consuming and expensive. Besides human annotators, though, companies often have a vast amount of information and knowledge sources available that can generate low-cost labels (e.g., a black-box model) or improve the learning process (e.g., a pre-trained model). We present a novel approach that enhances AL with weak supervision (WS) and transfer learning (TL) to reduce the annotation cost by leveraging these sources. Specifically, we consider a black-box model like a rule-based system as an error-prone and weakly-supervised annotator that inexpensively provides labels. We estimate its performance with an annotator model to decide whether a human annotation is required. Additionally, we utilize unlabeled internal and external data by transferring knowledge from a pre-trained model to the AL cycle. We sequentially investigate the impact of WS and TL on annotation cost and model performance in an AL cycle through a use case. Our evaluation shows that our approach can reduce annotation cost by 51% while achieving nearly identical model performance compared to a traditional AL approach.