Research interests and publications
Given some variable \(x \in X\) and \(y \in Y\) and functions \(f, g: X \times Y \to \mathbb{R}\), bilevel optimization is written as the following optimization problem:
\[\min_{x \in X} f(x, y^\star(x)) \quad \text{ subject to } \quad y^\star(x) \in \arg \min_{y \in Y} g(x, y).\]Stochastic bilevel optimization pertains to the case where the functions \(f,g\) are stochastic functions with stochastic oracles \(\xi\) and \(\psi\) such that
\[f(x, y) \triangleq \mathbb{E}_{\xi} F(x, y; \xi), \quad g(x, y) \triangleq \mathbb{E}_{\psi} G(x, y; \psi),\]which is common in AI/ML where the functions \(F, G\) are training/validation losses on some batch of the data.
Ram, P., Gray, A. G., Samulowitz, H. C., & Bramble, G. (2023). Toward Theoretical Guidance for Two Common Questions in Practical Cross-Validation based Hyperparameter Selection. SIAM Internation Conference on Data Mining. arXiv, paper
Gu, A., Lu, S., Ram, P., & Weng, L. (2022). Min-max Bilevel Multi-objective Optimization with Applications in Machine Learning. International Conference in Learning Representations. arXiv OpenReview
Zhou, Y., Ram, P., Salonidis, T., Baracaldo, N., Samulowitz, H., & Ludwig, H. (2022). Single-shot Hyper-parameter Optimization for Federated Learning: A General Algorithm & Analysis. International Conference in Learning Representations. arXiv OpenReview slides (notable-top-25%)
Zhang, Y., Sharma, P., Ram, P., Hong, M., Varshney, K. R., & Liu, S. (2023) What Is Missing in IRM Training and Evaluation? Challenges and Solutions. International Conference in Learning Representations. arXiv OpenReview
Zhang, Y., Yao, Y., Ram, P., Zhao, P., Chen, T., Hong, M. & Liu, S. (2022). Advancing Model Pruning via Bi-level Optimization. In Annual Conference on Neural Information Processing Systems. arXiv OpenReview
Gu, A., Lu, S., Ram, P., & Weng, L. (2022). Robust Multi-objective Bilevel Optimization With Applications In Machine Learning. In INFORMS Annual Meeting. webpage slides
Teng, Y., Choromanska, A., Campbell, M., Lu, S., Ram, P., & Horesh, L. (2022). Overcoming Catastrophic Forgetting via Direction-Constrained Optimization. In European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases. paper
Zhao, P., Ram, P., Lu, S., Yao, Y., Bouneffouf, D., Lin, X., & Liu, S. (2022). Learning to Generate Image Source-Agnostic Universal Adversarial Perturbations. In International Joint Conference on Artificial Intelligence. paper arXiv
Fan, C., Ram, P., & Liu, S. (2021). Sign-MAML: Efficient Model-Agnostic Meta-Learning by SignSGD. 5th Workshop on Meta-Learning at NeurIPS 2021 arXiv
Gu, A., Lu, S., Ram, P., and Weng, T.-W. (2021). Nonconvex min-max bilevel optimization for task robust meta learning. In Beyond first order methods in machine learning systems at ICML 2021. paper