Accelerated First-Order Optimization Algorithms for Machine Learning

Numerical optimization serves as one of the pillars of machine learning. To meet the demands of big data applications, lots of efforts have been done on designing theoretically and practically fast algorithms. This paper provides a comprehensive …

Accelerated Alternating Direction Method of Multipliers: an Optimal O(1/K) Nonergodic Analysis

The Alternating Direction Method of Multipliers (ADMM) is widely used for linearly constrained convex problems. It is proven to have an O(1/√K) nonergodic convergence rateand a faster O(1/K) ergodic rate after ergodic averaging, where K is the number …

Accelerated Proximal Gradient Methods for Nonconvex Programming

We extend APG for general nonconvex and nonsmooth programs by introducing a monitor that satisfies the sufficient descent property

Smoothed Low Rank and Sparse Matrix Recovery by Iteratively Reweighted Least Squared Minimization

We presents a general framework for solving the low-rank and/or sparse matrix minimization problems, which may involve multiple nonsmooth terms.