Kazuho Watanabe, Toyohashi University of Technology, JapanTitle:
Rate-distortion dimension and Bayesian learning coefficientAbstract:
Rate-distortion theory analyzes fundamental limits of lossy compression methods. In this study, we discuss the connection of rate-distortion theory to Bayesian learning. We formulate a rate-distortion problem by the distortion measure defined by the pointwise regret of the model parameter, and show that the generalized Bayesian posterior distribution appears as the optimal solution to the problem. We show the relationship between the asymptotic behavior of the rate-distortion function and the learning coefficient of Bayesian learning, which characterizes the model's generalization ability. Furthermore, we numerically demonstrate this relationship in a binomial mixture model as an example of latent variable models.