Efficient Priors for Scalable Variational Inference in Bayesian Deep Neural Networks

Specifying meaningful weight priors in Bayesian deep neural network (DNN) is a challenging problem, particularly for scaling variational inference to larger models involving high dimensional weight space. We propose MOPED (MOdel Priors Extracted from Deterministic DNN) method to choose informed weight priors in Bayesian DNN using Empirical Bayes framework.