Hi @yusri_dh. Since these are all tiny tensors, I would expect Pyro to be around 10x slower than Edward, but 100x slower surprises me. One thing that would speed up your model is to make a vectorized observation:

@yusri_dh I've updated the example by adding .expand_by(data.shape).independent(1). This should hopefully work in Pyro 0.2. If you're using Pyro 0.1.2 you might try dist.Bernoulli(f.expand(data.shape)).

Thank you @fritzo . Now the code is faster even though still slower than Edward. As you expect it is 10x slower than Edward (120 seconds).Why is pytorch slower than Edward in this case? Is it because of the backend, pytorch vs tensorflow?In what case, pyro will be faster than Edward because in Dustin Tran's blog it's said pyro will be faster in CPU than Edward?I want to use pyro because, in my old university server, it is difficult to install tensorflow and Edward (it needs a newer version of GCC).

PyTorch will be slower than Tensorflow for small tensor computations because Tensorflow can compile the computation graph and thereby avoid Python overhead. For larger tensor computations the two frameworks may be closer, though Tensorflow is still better at executing multiple tensor ops in parallel.

We're currently looking into two ways to speed up Pyro:1. First we're parallelizing gradient calculations to reduce variance. This can help in models with small tensors. Currently we only support parallelizing by hand (e.g. see test_enum.py), but we're working to automate this.2. PyTorch has a JIT compiler that can help eliminate Python overhead. This may speed up Pyro models in the future.