variational autoencoder
Canonical release
Tensorflow和Pytorch中变量自动编码器的参考实现。
我推荐Pytorch版本。它包括一个更具表现力的变异家族的示例，即反向自回归流。
变分推断用于拟合模型以二进制MNIST手写数字图像。推理网络（编码器）用于摊销数据点之间的推理和共享参数。可能性是通过生成网络（解码器）参数化的。
博客文章：https：//jaan.io/what-is-variational-autoencoder-vae-tutorial/
（Anaconda环境在environment-jax.yml中）
重要性抽样用于估计Hugo Larochelle的二进制MNIST数据集的边际可能性。测试集的最终边缘可能性为-97.10 NAT与已发布的数字相当。
$ python train_variational_autoencoder_pytorch.py --variational mean-field --use_gpu --data_dir $DAT --max_iterations 30000 --log_interval 10000
Step 0 Train ELBO estimate: -558.027 Validation ELBO estimate: -384.432 Validation log p(x) estimate: -355.430 Speed: 2.72e+06 examples/s
Step 10000 Train ELBO estimate: -111.323 Validation ELBO estimate: -109.048 Validation log p(x) estimate: -103.746 Speed: 2.64e+04 examples/s
Step 20000 Train ELBO estimate: -103.013 Validation ELBO estimate: -107.655 Validation log p(x) estimate: -101.275 Speed: 2.63e+04 examples/s
Step 29999 Test ELBO estimate: -106.642 Test log p(x) estimate: -100.309
Total time: 2.49 minutes
使用非平均场，更具表现力的后近似值（反向自回归流量，https://arxiv.org/abs/1606.04934），测试边缘log-likelihienhoodhienhoodhienhienhienhienhienyhienhions to升至-95.33 NATS：
$ python train_variational_autoencoder_pytorch.py --variational flow
step: 0 train elbo: -578.35
step: 0 valid elbo: -407.06 valid log p(x): -367.88
step: 10000 train elbo: -106.63
step: 10000 valid elbo: -110.12 valid log p(x): -104.00
step: 20000 train elbo: -101.51
step: 20000 valid elbo: -105.02 valid log p(x): -99.11
step: 30000 train elbo: -98.70
step: 30000 valid elbo: -103.76 valid log p(x): -97.71
使用JAX（Anaconda环境在environment-jax.yml中），以在Pytorch上获得3倍的速度：
$ python train_variational_autoencoder_jax.py --variational mean-field
Step 0 Train ELBO estimate: -566.059 Validation ELBO estimate: -565.755 Validation log p(x) estimate: -557.914 Speed: 2.56e+11 examples/s
Step 10000 Train ELBO estimate: -98.560 Validation ELBO estimate: -105.725 Validation log p(x) estimate: -98.973 Speed: 7.03e+04 examples/s
Step 20000 Train ELBO estimate: -109.794 Validation ELBO estimate: -105.756 Validation log p(x) estimate: -97.914 Speed: 4.26e+04 examples/s
Step 29999 Test ELBO estimate: -104.867 Test log p(x) estimate: -96.716
Total time: 0.810 minutes
JAX中的反向自回旋流动：
$ python train_variational_autoencoder_jax.py --variational flow
Step 0 Train ELBO estimate: -727.404 Validation ELBO estimate: -726.977 Validation log p(x) estimate: -713.389 Speed: 2.56e+11 examples/s
Step 10000 Train ELBO estimate: -100.093 Validation ELBO estimate: -106.985 Validation log p(x) estimate: -99.565 Speed: 2.57e+04 examples/s
Step 20000 Train ELBO estimate: -113.073 Validation ELBO estimate: -108.057 Validation log p(x) estimate: -98.841 Speed: 3.37e+04 examples/s
Step 29999 Test ELBO estimate: -106.803 Test log p(x) estimate: -97.620
Total time: 2.350 minutes
（平均场和反向自回旋流量之间的差异可能是由于几个因素所致，主要是在实施中缺乏卷积。在https://arxiv.org/pdf/1606.04934.pdf中使用了残留块，以使Elbo近距离接近-80 NATS。）。
python train_variational_autoencoder_tensorflow.pyconvert -delay 20 -loop 0 *.jpg latent-space.gif 
