Celestine Mendler-Dünner

Talks and Events

• upcoming
  Speaking at the MPG AI & Research event in Berlin
• 11/2024
  Speaking at the ETH Seminar on Law & Economics
• 09/2024
  Keynote talk at the Cornell Thought Summit on the Future of Survey Science
• 07/2024
  Invited tutorial on Performative Prediction at UAI with Tijana
• 07/2024
  Keynote speaker at the joint conference CAp+RFIAP in Lille
• 06/2024
  Speaking at the ELLIS Institute Opening
• 12/2023
  Invited talk at the NeurIPS Workshop on Algorithmic Fairness
• 11/2023
  Serving as Program Chair for EAAMO'23
• 05/2023
  Speaking at the Oxford Workshop for Social Foundations of Statistics and ML

Affiliations and Awards

Advising

Resources, Media and Recordings

• Powermeter is available in the Chrome store
• Invited talk NeurIPS workshop on Algorithmic Fairness Through the Lense of Time.
• Invited talk Oxford Workshop on Social Foundations of Statistics and Machine Learning
• Invited talk MIT Optimization Seminar
• Snap ML in the press: Forbes, The Register and EE Times writing about our research
• Snap ML has been integrated with the AI Toolkit for IBM Z and Watson ML.
• Snap ML available through PyPi: > pip install snapml (see Medium article)
• General audience interview for the Cyber Valley Podcast Series, see below.

Featured Research Projects

LLMs and Surveys Performative Prediction Power in Digital Economies Algorithmic Collective Action Scalable Learning Algorithms

LLMs and Surveys

As large language models increase in capability, researchers have started to conduct surveys of all kinds on these models---research questions rage from learning about biases and alignment of LLMs, to extracting information about human subpopulations and using them as a data source for social science research. In our work we carefully examine the validity of surveys as a method to extract population statistics from LLMs. We explain pitfalls, develop tools for systematic testing the capabilities of LLMs to emulate human outcomes, and design statistically rigorous methods to integrate them into the research pipeline.

Evaluating LLMs as risk scores

A Python library to systematically translate the ACS survey data into natural text prompts to benchmark LLM outputs against US population statistics. Individual prediction tasks are non-realizable and provide insights into model's ability to express natural uncertainty in human outcomes.

Performative Prediction

Predictions when deployed in societal systems can trigger actions and reactions of individuals. Thereby they can change the way the broader system behaves -- a dynamic effect that traditional machine learning fails to account for. To formalize and reason about performativity in the context of machine learning, we have developed the framework of performative prediction. It extends the classical risk minimization framework in that it allows the data distribution to depend on the deployed model. This inherently dynamic viewpoint leads to new solution concepts and optimization challenges, and it brings forth interesting connections to concepts from causality, game theory and control.

Performative Prediction

• Performative Prediction. ICML 2020.
• Survey paper. Arxiv 2023.
• Tutorial. UAI 2024.

Power in Digital Economies

Algorithmic predictions play an increasingly important role in digital economies as they mediate services, platforms, and markets at societal scale. The fact that such services can steer consumption and behavior is a central concern in modern anti-trust. I am exploring how the concept of performativity can help quantify economic power in digital economies and support digital market investigations. Intuitively, the more powerful a firm the more performative their predictions, a causal effect strength we can measure from observational and experimental data.

Performative power of online search

A Chrome extension that measure how content arrangements impact user click behavior through randomized experiments.

• Project website at https://powermeter.is.tue.mpg.de/
• Experimental insights to be published at NeurIPS 2024

Algorithmic Collective Action

Digital platforms critically rely on data provided by individuals. Thus data offers a lever to the participants for countering power imbalances and gaining some control over the system. It can be seen a new technology specific alternative to traditional strikes. We study different learning settings and quantify the critical mass of individuals that need to be mobilized to achieve concrete goals. The effecitveness of strategies is closely related to performativity of the platform's algorithm. Research questions encompass the design of strategies, challenges of coordination, as well as connection to labor markets and collective action theory in political economy.

Algorithmic collective action

• Theoretical framework. ICML 2022.
• Application to sequential recommender systems. NeurIPS 2024.
• A collection of documented use-cases. Github.

Scalable Learning Algorithms

When building societal-scale machine learning systems, training time and resource constraints can be a critical bottleneck for dynamic system optimization. Efficient training algorithms that are aware of distributed architectures, interconnect topologies, memory constraints and accelerator units form an important bilding block towards using available resources most efficiently. As part of my PhD research, we demonstrated that system-aware algroithms can lead to several orders of magnitude reduction in training time compared to standard system-agnostic methods. Today, the innovations of my PhD research form the backbone of the IBM Snap ML library that has been integrated with several of IBM's core AI products.

IBM Snap Machine Learning

Snap ML is a library that provides resource efficient and fast training of popular machine learning models on modern computing systems.
>400k downloads on PyPi
https://www.zurich.ibm.com/snapml/