March 28, 2021

PyMC3, Pyro vs (Py)STAN

Hi,

Welcome to the first Naive Bayesians newsletter.

PyMC3, Pyro vs (Py)STAN

There are at least 8+ probabilities programming frameworks out there. The 3 most important ones to consider are PyMC3, (Py)Stan, and Pyro.

Very rough tldr:

• (Py)Stan:
  • The statisticians choice.
  • Has excellent documentation and examples.
  • Built initially by Andrew Gelman, et. al.
  • Strong support from the Bayesian academic community.
• Pyro:
  • The Deep Learning Engineer’s choice.
  • Built by Uber.
  • Built on top of PyTorch Geared towards applying bayesian inference on neural networks
• PyMC3:
  • The Generalist’s choice.
  • Built on top of Theano (an older neural network API) but moving to JAX soon.
  • It sits between (Py)STAN and Pyro.
  • IMO a much simpler API to start learning and building Bayesian models. Strong community forums.
• All these frameworks help you achieve similar goals. So, choosing one-vs-the-other isn't a big deal for building your first few bayesian models.

Recommendation:

• Start with PyMC3, then move on to Pyro for neural networks (esp. if you’re familiar with PyTorch). Forget about everything else.

References:

• Benchmarks/Overview: A tour of probabilistic programming language APIs: https://colcarroll.github.io/ppl-api/

• To go deeper: Anatomy of a Probabilistic Programming Framework: https://eigenfoo.xyz/prob-prog-frameworks/