Theoretical Facts about VAEs

  • \(\beta\)-VAE
    • \(\beta\)-VAE objective is equivalent to a standard ELBO with the alternative prior \(f_{\beta}(\mathbf{z}) \propto p(\mathbf{z})^{\beta}\) (Mathieu et al. 2018):
      • for Gaussian \(p(\mathbf{z})\):
        • \(\beta\)-VAE simply anneals the latent space by \(1/\sqrt{\beta}\), i.e., \(f_{\beta}(\mathbf{z}) = \mathcal{N}(\mathbf{z};0,\Sigma/\beta)\).
        • \(\beta\)-VAE objective is invariant to rotations of the latent space.

References

Mathieu, Emile, Tom Rainforth, N. Siddharth, and Yee Whye Teh. 2018. “Disentangling Disentanglement in Variational Autoencoders.” arXiv:1812.02833 [Cs, Stat], December. http://arxiv.org/abs/1812.02833.