AIM Seminar: Cyclic Block Optimization: How they work, why they work, and where they work

Abstract: When facing challenging tasks in life, one natural strategy is to solve simple sub-tasks one by one and hope that it eventually lead to somewhere better. Many challenging optimization problems in machine learning and scientific computing follow this approach—optimizing a small block of parameters cyclically, one at a time. Notable examples include Sinkhorn’s algorithm for computing optimal transport maps and Schrödinger bridges, as well as alternating least squares and multiplicative updates for matrix and tensor factorization. In this talk, we will explore the principles behind these methods (block majorization-minimization), why they often require less parameter tuning than first-order gradient-based approaches (via second-order analysis), and their recent extensions to Riemannian manifolds, with new results in Wasserstein variational inference.
This talk is based on recent works with Yuchen Li (UW), Joowon Lee (UW), Laura Balzano (Michigan), Deanna Needell (UCLA), and Sumit Mukerjee (Columbia).

Contact: Laura Balzano