Variational Inference: ELBO Computation
Sequence Models & Generative Models DS practice problem on Onlearn.
Difficulty: hard.
Topics: Variational Inference: ELBO Computation, Kullback-Leibler Divergence, Evidence Lower Bound, Reparameterization Trick, Mean-Field Assumption, Log-Likelihood Gradient, Probabilistic Graphical Models, Bayesian Inference, Information Theory, Optimization Theory, Deep Generative Modeling, Variational Approximation, Monte Carlo Methods, Latent Variable Models, Stochastic Gradient Estimation, Divergence Measures.
Implement the Evidence Lower Bound (ELBO) for variational inference. In simple terms: we want to figure out hidden variables (z) from observed data (x), but the exact calculation is too hard. Instead, we approximate with a simpler distribution (q) and measure how good our approximation is using ELBO. Higher ELBO means better approximation. You'll use Monte Carlo sampling (taking many random samples) to estimate the ELBO value.