Multi-Objective Dynamics of Learning
2026 – ongoing
Learning is rarely about a single goal. A model is asked to be accurate and fair, to solve several tasks at once, to serve different clients, domains, or modalities, or to balance competing notions of quality. Even when one loss is written down, it is often an implicit compromise among many. Yet we understand learning almost exclusively through a single-objective lens: objectives are merged into one scalar loss before training starts, and the trade-offs among them are only examined, if at all, once training has finished.
This research line treats learning itself as a multi-objective process. The central question is not only which trade-off a trained model ends up at, but how the trade-offs are formed during training. Objectives compete and cooperate through the representation they share: each one pulls the learned space toward the features it needs, and whatever the representation captures early on shapes what every objective can achieve later. When this competition is left unmanaged, some objectives take over the shared space and others are starved, which leads to fronts that are worse for everyone. When it is understood, the same process can be guided toward representations that are useful to all objectives at once, moving the whole set of solutions closer to the ideal point instead of merely sliding along a fixed trade-off.
The goal is therefore twofold: 1. Understanding: Describe how multiple objectives shape the learning trajectory, including which features are learned, in what order, and for whose benefit, and how this is reflected in the geometry of the attainable trade-offs. 2. Steering: Use that understanding to conduct learning so that the learned space improves every objective, through training signals, optimizers, and formulations that are aware of the multi-objective structure from the first step rather than only at the end.
The project builds on tools from multi-objective optimization (Pareto sensitivity, knee solutions, bilevel and game-theoretic formulations) and from the theory of feature learning in neural networks. It brings these into the training loop, where models are never at equilibrium and the relevant questions are dynamic.
Key Research Topics:
- Multi-Objective Learning Dynamics: Characterizing how objectives jointly shape a shared representation during training, including when they reinforce each other, when one dominates, and how early imbalances compound into lasting losses for some objectives.
- Geometry of Trade-offs Along Training: Extending Pareto sensitivity and front geometry beyond converged solutions, so that the shape of the attainable trade-offs can be measured and interpreted while learning is still in progress.
- Representations that Serve All Objectives: Identifying conditions and mechanisms under which a learned space improves every objective at once, connecting multi-objective structure to diversity, robustness, and generalization of representations.
- Multi-Objective-Aware Optimization: Designing training procedures that handle several objectives natively, including multi-objective adaptive optimizers (e.g., extensions of Adam), online adaptation of preferences, and single-pass methods that recover entire fronts at the cost of standard training.
- Theory–Experiment Bridge and Open Tools: Building controlled benchmarks with known ground-truth features, together with an open-source library for scalarization, sensitivity analysis, and front evaluation, so that claims about how objectives shape learning can be tested directly.
