Consensus-based High Dimensional Global Non-convex Optimization in Machine Learning - Shi JIN - Institute of Natural Sciences e Shanghai Jiao Tong University

We introduce a stochastic interacting particle consensus system for global optimization of high dimensional nonconvex functions. This algorithm does not use gradient of the function thus is suitable for non-smooth functions.
We prove, for fully discrete systems, that under dimension-independent conditions on the parameters, with suitable initial data, the algorithms converge to the neighborhood of the global minimum almost surely. We also introduce an Adaptive Moment Estimation (ADAM) based version to significantly improve its performance in highspace dimension.

• Locandina (1 MB)