Bayesian Neural Networks

Short summary

An academic overview of Bayesian neural networks adapted from an MS thesis, covering the motivation (underspecification in DNNs), the posterior predictive distribution, and several approximation methods. It discusses factored Gaussian priors, the infinite-width Gaussian process limit, Monte Carlo dropout as a simple Bayesian approximation, Bayesian last-layer methods, and scalable MCMC techniques like SGLD. The content is mathematically dense and assumes deep familiarity with neural network training and Bayesian inference.

• •Bayesian model averaging over multiple parameter settings can improve accuracy and uncertainty representation in DNNs
• •Monte Carlo dropout offers a simple approximation by keeping dropout active at test time and averaging K sampled models
• •Scalable MCMC methods like SGLD enable posterior approximation using mini-batches, addressing the computational bottleneck of standard HMC

Generated with AI, which can make mistakes.