Bayesian optimization: AutoML and hyperparameter tuning

• 강사 : 최승진 (정보과학회 인공지능소사이어티)
• 일시 : 2019. 9. 5 (목) 17:00~19:00

We often encounter a problem in practice where the objective to be optimized is a black-box function, i.e., a closed-form expression of which is not given but expensive evaluations at query points are available. Exemplary problems are hyperparameter optimization for deep neural networks and automated machine learning (autoML) where data preprocessing, algorithm selection are chosen automatically to output the best performance. Bayesian optimization provides a principled and efficient approach to the black-box optimization, in terms of the number of function evaluations required. Bayesian optimization places a prior over the objective function and combines it with the likelihood to calculate the posterior function. This permits determining where next to sample from the objective function, which is done by the acquisition function maximization, balancing exploration and exploitation. In this talk, I introduce an underlying idea of Bayesian optimization, including Gaussian process regression and various acquisition functions such as PI, EI, and GP-UCB. If time permitted, I will discuss a few recent issues on Bayesian optimization that you might ask.