My research lies at the interface between optimization, statistics and machine learning. I develop and analyze algorithms for learning problems, in order to explain their observed behavior and to provide actionable guidelines. My current work focuses on the long-run behavior of stochastic gradient methods and on distributionally robust learning.
Most machine learning models are trained by stochastic gradient descent (SGD) or one of its variants. With a finite training budget, these methods do not exactly minimize the training loss. Instead, they output a random model whose probability distribution depends on the loss landscape and on the gradient noise. Using large deviations theory, we characterize this long-run distribution, as well as the time needed to reach a global minimizer in non-convex landscapes. We have recently extended this analysis to learning in games.
Wasserstein distributionally robust optimization (WDRO) trains models that perform well on all probability distributions close to the empirical one, in the sense of optimal transport. We study the statistical guarantees of this approach, notably exact generalization bounds. Regularization makes these problems tractable, and we have applied them to risk control in prediction and to shape optimization. To make these models easy to use in practice, we develop the Python library skwdro.
Optimal transport maps are used to compare and align datasets. We study how to estimate these maps from samples under differential privacy constraints, and we characterize the minimax rates of this problem.
Nonsmooth regularizers, such as the ℓ1 norm, induce structure in the solutions of learning problems (sparsity, low rank). Proximal methods identify this structure after a finite number of iterations. My habilitation studies how to detect and exploit this structure, for instance to accelerate algorithms with Newton steps on the identified manifold or to reduce the communications of distributed methods.
When data and computations are spread over several machines, the agents must cooperate although communications are delayed, costly or intermittent. I have designed and analyzed asynchronous, randomized and delay-tolerant algorithms for this setting, from gossip and ADMM methods during my PhD to multi-agent online learning.
Adversarial training and learning in games lead to variational inequalities rather than minimization problems. We have analyzed extragradient and optimistic methods in stochastic settings, with adaptive step-sizes, as well as the convergence rates of Bregman proximal methods.
Member of the scientific council of the PGMO program of the Fondation Hadamard
Elected member of the liaison committee of SMAI-MODE, the optimization and decision group of the French Society for Applied Mathematics; in charge of the monthly MODE newsletter (about 400 subscribers)
Elected member of the MALIA group (machine learning and AI) of the French Statistical Society (SFdS)