INCHER & Zell Colloquium on 17. December 2025: "The inductive bias of deep neural networks toward feature learning" by Dr. Jakob Heiss

Lecture by Dr. Jakob Heiss, University of California, Berkeley, Department of Statistics, USA

Abstract: Machine Learning (ML) models rely on the assumption that similar inputs ($x$) correspond to similar labels ($y$) to make predictions on unseen data. Crucially, the definition of "similar" is dependent on the feature representation. The hidden layers of Deep Neural Networks (DNNs) often learn feature transformations, $h(x)$, that yield a practically meaningful notion of distance between inputs. For example, Large Language Models (LLMs) map the same sentence in different languages to similar representations in their embedding space. This ability to learn useful features often enables transfer learning and multi-task learning. My talk will provide a theory-inspired intuition for why this feature learning happens and, critically, when it provably does not.

Furthermore, an application of deep learning to market design will be presented, which inspired the Machine Learning-powered Course Match (MLCM): an ML-based mechanism for eliciting student preferences and assigning university courses.
Keywords: Representation Learning, Feature Learning, Multi-task Learning, Transfer Learning, Combinatorial Assignment

Dr. Jakob Heiss has since 2024 been a Postdoctoral Scholar with Prof. Bin Yu in the Yu Group at UC Berkeley, focusing on Deep Learning Theory and Uncertainty Quantification. He completed his Ph.D. (2019-2024) advised by Prof. Josef Teichmann in the Department of Mathematics at ETH Zurich, and was affiliated with the ETH AI Center. His research centers on the mathematical theory of deep learning algorithms, including inductive bias, multi-task learning, and network compression. He also works on Uncertainty Quantification, Neural Jump ODEs for irregularly observed time series, and the application of deep learning to market design (preference elicitation)

The INCHER lectures  2025 are hybrid events.
If you wish to participate via Zoom please register at koch[at]incher.uni-kassel[dot]de

Venue: International House, Mönchebergstrasse 11a, University of Kassel 
34125 Kassel