Advances in Convex Optimization Algorithms for Data-Driven Machine Learning Applications

Abstract

Convex optimization forms a fundamental pillar of modern machine learning and data science,
offering guarantees of global optimality, theoretical stability, and interpretability. This paper presents a unified
and reproducible framework that connects core convex analytical concepts—including convex sets, smoothness,
duality, proximal calculus, and monotone operator splitting—with practical algorithmic implementations for
large-scale learning.
We investigate five representative optimization strategies: proximal gradient descent (ISTA), accelerated
variants (FISTA), variance-reduced stochastic methods (SVRG, SAGA), and the Alternating Direction Method
of Multipliers (ADMM) for distributed and constrained optimization. Python/CVXPY-based experiments on
sparse regression, logistic classification, and portfolio optimization demonstrate accelerated convergence, im-
proved sparsity recovery, and enhanced scalability over classical gradient methods. Numerical evaluations
further quantify performance trade-offs across regularization strength, dataset size, and conditioning, validat-
ing theoretical convergence trends.
By integrating mathematical foundations with empirical verification, this work provides a transparent
benchmarking workflow and a cohesive perspective on convex optimization in machine learning. The accom-
panying open-source implementation strengthens reproducibility and serves as a reference platform for future
research on scalable convex learning and federated optimization.