LORENZO ROSASCO - UNIVERSITA DEGLI STUDI DI GENOVA - UNCONVENTIONAL REGULARISATION FOR EFFICIENT AND SUSTAINABLE MACHINE LEARNING

Classic algorithm design is based on penalising or imposing explicit constraints to an empirical, objective function, which is eventually optimised. In practice, however, a number of different algorithmic solutions are employed. Their effect on final performance is hard to assess a priori and typically done empirically.
In this talk, Prof. Rosasco will consider a linear least squares framework and will take a regularisation perspective to understand the effect of two commonly used ideas: sketching and iterative optimisation. This analysis will highlight the role and the interplay of different algorithmic choices, including training time, step and mini-batch size, and the choice of sketching, among others. Indeed, one can view all these choices as controlling a form of “algorithmic regularisation”. The obtained results provide new and practical guidelines for algorithm design. They suggest that optimal statistical accuracy can be achieved while dramatically improving computational efficiency. Theoretical findings will be illustrated in the context of large scale kernel methods, where Prof. Rosasco and co-authors developed the first solvers able to scale to millions of training points.
(This is a joint event with SmartData@poliTO)

• Poster (1 MB)