Stochastic Compositional Kernel Estimation for Gaussian Process Models
Résumé
In kernel-based learning, the choice of kernel can greatly impact the predictive performance of the model. Kernel selection is computationally intensive. However, various methods have been suggested for its optimal estimation. For example, exhaustive, grid, randomized and nonparametric search methods are few notable mentions. The effectiveness of these approaches is dependent on the intricacies of the data and the frameworks in which they operate. For instance, in Gaussian models, the dimension of the covariance matrix presents a challenge for suitable kernel assessment. In the case of variational and MCMC-based models, the time complexity required for the ELBO and posterior convergence hinders the implementation of optimal search. As such, in addition to the respective strategy, a computationally efficient exploration should take into account the limitations of the underlying model. This paper proposes a stochastic compositional kernel search algorithm. It follows a randomized point selection and cross validation in building the gaussian model. The root-mean squared error (RMSE) is applied as a criterion to evaluate the models and the optimality of the returned mixtures in explaining the given observations. We tested the algorithm on real and synthetic data. The experiments showed, the design iteratively offers possible kernel combinations by following the path with the least RMSE score. The sparsity in model building and the stochastic approach for kernel selection has afforded the algorithm a computational advantage over other exhaustive methods. As such, it can be used as an alternative technique for a suitable kernel selection that best explain the data.