Benchmarks

To run the benchmarks, the fastest way is to use the docker image.

The blockchain snapshot is in contract_data in the Github Repo and the benchmark results are in benchmarks/results.json. The benchmarks were run using a 5-minute timeout - using a longer timeout will give fewer incomplete executions.

In the container, run batch_run.py after activating the virtualenv. Results are in results.json once the benchmark completes.

The benchmarks take a long time and a lot of RAM in any but the largest of clusters, beware.

Benchmark Results

Bug

Occurrences

Callstack Bug

4792

Reentrancy

3742

Time Dependancy

165

Concurrency

490

Total

11839

(Timeout)

5715

The results are different from those in the paper. This is an improved version of the tool, with fewer exceptions.