ELBO
E835245
ELBO (Evidence Lower Bound) is an objective function used in variational inference to approximate complex probability distributions, particularly in variational autoencoders and related Bayesian models.
Statements (49)
| Predicate | Object |
|---|---|
| instanceOf |
objective function
ⓘ
variational inference concept ⓘ |
| alternativeForm | log p(x) - KL(q(z|x) || p(z|x)) ⓘ |
| bounds | log marginal likelihood from below ⓘ |
| canBe |
decomposed into sum over data points
ⓘ
estimated with Monte Carlo samples ⓘ |
| centralRoleIn | training of variational autoencoders (VAEs) ⓘ |
| definedOver |
latent variables
ⓘ
observed variables ⓘ variational distribution ⓘ |
| dependsOn |
model parameters
ⓘ
variational parameters ⓘ |
| fullName | Evidence Lower Bound ⓘ |
| hasTerm |
reconstruction term
ⓘ
regularization term ⓘ |
| introducedInContextOf | variational methods in Bayesian statistics ⓘ |
| isLowerBoundOn | log p(x) ⓘ |
| mathematicalDomain |
machine learning
ⓘ
probability theory ⓘ |
| maximizationEquivalentTo | minimizing KL divergence between variational posterior and true posterior ⓘ |
| optimizedBy |
reparameterization trick
ⓘ
stochastic gradient descent ⓘ stochastic variational inference ⓘ |
| property |
non-convex in general for deep models
ⓘ
tighter ELBO implies better approximation to true posterior ⓘ |
| regularizationTermOften | KL(q(z|x) || p(z)) ⓘ |
| relatedTo |
Kullback–Leibler divergence
NERFINISHED
ⓘ
log evidence ⓘ marginal likelihood ⓘ negative variational free energy ⓘ variational free energy ⓘ |
| typicalForm | E_q[log p(x,z)] - E_q[log q(z)] ⓘ |
| usedFor |
approximating complex probability distributions
ⓘ
learning generative models ⓘ optimizing variational distributions ⓘ training probabilistic models with latent variables ⓘ |
| usedIn |
Bayesian inference
ⓘ
Bayesian neural networks NERFINISHED ⓘ amortized variational inference ⓘ approximate Bayesian inference ⓘ black-box variational inference ⓘ deep generative models ⓘ latent variable models ⓘ probabilistic programming ⓘ variational autoencoder ⓘ variational inference ⓘ |
| variant |
beta-ELBO
ⓘ
hierarchical ELBO ⓘ importance-weighted ELBO ⓘ |
Referenced by (1)
Full triples — surface form annotated when it differs from this entity's canonical label.