Our good friend biglesp put the Raspberry Pi 2 to the test the last time around, though I feel that he missed a good number of possible benchmarks to run on the hardware. With that in mind, I'm going to test the main boards of the Raspberry Pi that I can get my hands on and not only run the main benchmarks Les ran, but also a good few more thanks to Roy Longbottom's benchmark collection. I also have some approximate information on idle power usage, which everyone appears to be interested in.

One technical capability that I would have loved to put through its paces with Roy's benchmarks is the OpenGL driver that the Pi Foundation have made available. Unfortunately the Roy benchmarks are currently not compatible with it, so I guess it has just been testing against the binary blob for now. Regardless, the VideoCore is one part of the ARM processor that practically has not changed from the early Model B to the most recent version of the Raspberry Pi.

Modelling for the Camera

It does not hurt to go over the versions of Raspberry Pi again, and here are the models that I tested:

If you want more information on the specifications you can check the comparison chart linked in the Raspberry Pi space. What we're mainly concerned with is the processor, power usage and somewhat also RAM, however the RAM runs at 400Mhz for each board so that may make little difference.

What does make somewhat of a difference is that the Raspberry Pi 3 is an ARMv8 Processor with 64bit support, however, Raspbian at the time of writing does not have 64bit support. In part this is so that it is backwards compatible with the previous Raspberry Pi. There is yet to be a 64bit Repo. Debian has been building up its 64bit support over a period of years, and potentially someone could compile together a kernel and binaries for a 64bit Jessie Debian for the Raspberry Pi 3. Go on, accept the challenge!

Taste Testing the Pi

Before I get into the software used for the benchmarking, I decided to run a few commands to get the processor information and supported instructions:

What is pretty clear with cpuinfo is that it identifies the Raspberry Pi 3 processor as an ARMv7, this appears to be a failure in the software for now but this may change with a kernel/firmware update to ARMv8. What is interesting is that we have a new feature: crc32. I also never realised that "java" was actually a feature on the Model B+ processor chip and that's one that has been lost over time (some would say that is no love lost).

lscpu - Raspberry Pi 3 Model B

lscpu - Raspberry Pi 2 Model B

lscpu - Raspberry Pi 1 Model B+

Architecture: armv7l

Byte Order: Little Endian

CPU(s): 4

On-line CPU(s) list: 0-3

Thread(s) per core: 1

Core(s) per socket: 4

Socket(s): 1

Model name: ARMv7 Processor rev 4 (v7l)

CPU max MHz: 1200.0000

CPU min MHz: 600.0000

Architecture: armv7l

Byte Order: Little Endian

CPU(s): 4

On-line CPU(s) list: 0-3

Thread(s) per core: 1

Core(s) per socket: 4

Socket(s): 1

Model name: ARMv7 Processor rev 5 (v7l)

CPU max MHz: 900.0000

CPU min MHz: 600.0000

Architecture: armv6l

Byte Order: Little Endian

CPU(s): 1

On-line CPU(s) list: 0

Thread(s) per core: 1

Core(s) per socket: 1

Socket(s): 1

Model name: ARMv6-compatible processor rev 7 (v6l)

CPU max MHz: 700.0000

CPU min MHz: 700.0000

With the lscpu command we can see that the Pi 3 processor is perhaps still not being identified properly, but at least that it, like the Pi 2 before it, can perform cpufreq scaling (which is also shown when issuing the lshw command). We can also clearly see that we have a confirmed, rather gladly, four core processor with the Pi 2 and Pi 3. Phew!

If you want to run these commands yourself on your Raspberry Pi with Raspbian then you can make sure you have them installed thusly:

sudo apt-get update

sudo apt-get install lscpu cpuinfo lshw

From a terminal window either within your desktop environment or from pressing CTRL-ALT-F1 (to F7, typically).

Power Usage

You can find some strange little devices that will plug in line with your USB hardware and it will tell you the most useful of things, such as how much power is being drawn! So I thought "sweet! Let's see how much power these draw while idle!"

Raspberry Pi 3 Model B

Raspberry Pi 2 Model B

Raspberry Pi 1 Model B+

5.19

V

1.141

W

0.22

A

00006

mAh

5.19

V

1.038

W

0.20

A

00102

mAh

5.19

V

0.986

W

0.19

A

00003

mAh

Let me give these values some context, each Raspberry Pi was set to boot to the terminal, so that the X windows environment was not running. The only devices connected were a HDMI to DVI adapter to a 19" Widescreen monitor, a Dell USB keyboard, a 16gByte Class 10 microSD card and the power supply, which was providing 5 Volts, 2 Amps. There was no ethernet cable plugged in (though I can note that when it was, the power usage went up in all cases). We can see from the charts that the Pi 3 is pulling more power at idle than its predecessors, it is likely worth noting that the on board WiFi adapter of the Pi 3 was active, though it was not associated with an access point. The Bluetooth 4.1 adapter status was not intentionally active, as there was no driver loaded or software to use it. So it is likely the increase in power usage is due to the WiFi chip.

SysBench

Now this software has been around since 2004, it was originally intended for input/output (io) file operations and database benchmarking. Thanks to being open source it gradually developed into an almost all-round system benchmark which also includes aspects of processor testing as well as IO and databases.

SysBench's processor tests verify prime numbers by going through all possible divisions and only being satisfied when the result is zero. This does mean that it does not test all features of the processor, except for raw number crunching. SysBench was ran with the following parameters:

--num-threads - This is the number of processes to run, in the tests we run 1 thread and then 4 threads, this means that it will create 1 or 4 processes and run one process per core. Since the Raspberry Pi 1 Model B+ has one core it made sense to run a one core test on each model of Raspberry Pi for fairness alongside running 4 threads.

--test=cpu - This parameter ensures we are only testing the processor, as mentioned previously SysBench can perform other tests, too

--cpu-max-prime - This is the maximum prime number value we want to calculate up to.

--validate - This ensures that the results we have returned are valid

run - The software can emulate or test rather than actually perform the requested benchmark, so we want to tell it to actually run it.

sysbench with 1 thread - Raspberry Pi 3 Model B

sysbench with 1 thread - Raspberry Pi 2 Model B

sysbench with 1 thread - Raspberry Pi 1 Model B+

Total time 477.0617 s

per request

statistics

min

47.69

ms

avg

47.7

ms

max

49.91

ms

diff between min and max

2.22 ms

Total time 768.6476 s

per request

statistics

min

76.42

ms

avg

76.86

ms

max

82.15

ms

diff between min and max

5.73 ms

Total time 1318.933 s

per request

statistics

min

131.59

ms

avg

131.89

ms

max

300.23

ms

diff between min and max

168.64 ms

With the Pi 3 versus the Pi 1, the results show about a 94% total speed difference, compared to the Pi 2 there is a 47% speed difference on the total time taken. What I find most interesting about these results, though, is the difference between the minimum and maximum time taken. The Pi 3 shows a 194% speed improvement over the B+ and an 88% speed improvement over the Pi 2. So while the overall time may not feel that big of a deal, per request it is performing them a lot faster. There's also evidence that it is overall significantly faster than the B+.

sysbench with 4 threads - Raspberry Pi 3 Model B

sysbench with 4 threads - Raspberry Pi 3 Model B

sysbench with 3 threads - Raspberry Pi 1 Model B+

Total time 119.4716

per request

statistics

min

47.69

ms

avg

47.78

ms

max

59.04

ms

diff between min and max

11.35 s

Total time 191.8972 s

per request

statistics

min

76.38

ms

avg

76.74

ms

max

101.45

ms

diff between min and max

25.07 ms

Total time 1321.493 s

per request

statistics

min

412.94

ms

avg

528.54

ms

max

573

ms

diff between min and max

160.06 ms

This test is an unfair comparison with the Pi 1 B+ since it only has one processor core, so it is not surprising that it takes significantly longer than the Pi 2 and 3. The Pi 2 closes the gap a bit closer when working with all four cores here than with the single core test though the Pi 3 still outperforms by 75% with the difference between the min and max time it takes for the requests, overall there is still only a 47% speed difference which is almost half.

Overall the Pi 3 comes out on top as faster, though it would be interesting to see how the Pi 2 compares if it was over clocked. Still, something to remember is that computing prime numbers is just one function of what this processor is capable of.

nBench

I thought it would be good to include this for retro' computing sake, especially considering that the Raspberry Pi is intended to be going back to the roots of learning how to code and programme with a capable hands-on computer platform. nBench has been around for so long that you can compare these benchmark results against older processors such as the 386 and even 486 based processors right up to the Intel Core i7. It can even run on Android.

The software is intended to test three main components of the processor, the capabilities of the CPU (Central Processing Unit), FPU (Floating Point Unit) and memory system. You can find more information on this site. Once you have compiled nBench then it is simply ran with a single command of what the program is called. In the results, the higher the number, the better, as it is the number of iterations it can perform per second, summarised.

nbench - Raspberry Pi 3 Model B

nbench - Raspberry Pi 2 Model B

nbench - Raspberry Pi 1 Model B+

memory index

7.105

integer index

8.976

floating-point index

7.601

memory index

4.186

integer index

5.812

floating-point index

4.526

memory index

2.501

integer index

3.208

floating-point index

1.884

Since I threw this benchmark in for fun, here are some other processors that have taken the same benchmark, to put these in perspective:

nbench - AMD K7 Thunderbird

nbench - Pentium 3 900Mhz

nbench - LG Optimus GT540

memory index

9.473

integer index

6.744

floating-point index

12.501

memory index

3.930

integer index

3.649

floating-point index

9.631

memory index

1.171

integer index

1.691

floating-point index

0.489

It is totally not fair to compare the Raspberry Pi processor to these chips, except for the LG Optimus, that is using an ARM processor, however I found it interesting that the Broadcom chip has its own capabilities and areas where it excels in comparison to these older processors, for example the integer capability on the Pi 3 excels against the old AMD K7 processor and both the Pi 3 and Pi 2 processors give the Pentium 3 a run for its money, though Intel are typically known for strong Floating Point arithmetic, it is typically known that GPU (Graphical Processing Units) are better for this these days and using the VideoCore in the Broadcom chip would likely perform better.

That and these latter processors were intended for desktop computers. There are are a lot of numbers which the software outputs, like the SysBench benchmark I have attached the spreadsheet containing all of the values to this blog post that you can find at the bottom.

MemTester

This software is mainly intended to diagnose or test your system RAM (Random Access Memory). You can read more about it in its man page on linux. In this test I have put a limit on it, it is to only test 256mByte of RAM. This helps to make it a fair test across the different models of Raspberry Pi. If you are not aware, the Raspberry Pi shares its system RAM with the VideoCore processor, and it is not really recommended to deny the VideoCore processor from using any of the RAM available. So that means we cannot test the entire 512mByte or 1gByte of RAM available on the Pi 1 or Pi 2 and 3.

Memtester by itself does not time how long it takes to check the amount of memory we specify. However, we can set how many times it does it. There is also a command in Linux called 'time' which, when used in conjunction with a command, tells us how long it has taken for the command to run. Using this simple command we can check how long it has taken to test 256mByte of RAM on each Raspberry Pi:

sudo time memtester 256M 1

Raspberry Pi Board

Time Taken

1 Model B+

76 minutes 23.296 seconds

2 Model B

23 minutes 39.07 seconds

3 Model B

8 minutes 37.078 seconds

The numbers speak for themselves, even though the RAM speed is pretty much the same being at least 400Mhz, evidently the limit is largely with the processor and we are seeing significant speed increases. In this case the Pi 3 is more than 50% faster than the Pi 2 at allocating and accessing RAM.

Roy's Benchmark Collection

There are a lot of tests. I mean, a lot. They range from whestone, ARMv6 specific tests, to ARMv7 and NEON tests. They also cover OpenGL and also memory tests deeper than by the ones previously in this blog post. So rather than go through every single test, I have compressed them together and attached them to this blog post.

I cannot cop out entirely though, can I? Well, I could, but I am not going to. Here are a few highlight comparison benchmarks from the text files that you can see as worthwhile to read:

Memory Reading Speed Test 32bit Version 4 (memSpeedPiA6)

Memory KBytes Used

Double MB/S - Raspberry Pi 3 Model B

Double MB/S - Raspberry Pi 2 Model B

Double MB/S - Raspberry Pi 1 Model B+

8

16

32

64

128

256

512

1024

2048

4096

8192

1523

1641

1523

1524

1524

1525

1409

1094

1075

1023

1071

1015

1015

1016

930

853

853

682

393

310

301

307

602

538

292

262

176

142

132

134

134

136

134

Now there is no doubt here that the Pi 3 is performing really well, what I am most interested in is the scaling from where it starts to drop off in performance. On both the Pi 2 and the Pi 1 we can clearly see that performance reduces at around the 32/16Kbyte used mark, but with the Pi 3 we do not start seeing degradation in speed until we hit 512kByte, though 16kByte is obviously a sweet spot even when we hit 1024kByte and above the drop is not as consistently severe as it is with the Pi 2. So there is obviously a fundamental change in how the hardware handles memory or pushes its numbers around.

NEON Speed Test v1.0

Raspberry Pi 3 Model B

Raspberry Pi 2 Model B

NEON technology was added with the Raspberry Pi 2, so it is not possible to test it on the Pi 1 Model B+, however I had not seen any benchmarks that had actually put it to the test until now. This increase in speed result should mean that the Pi 3 is faster at handling calculations relating to video, vector graphics rendering (so gaming and 3D) and potentially audio processing. Again it is interesting to see the point at which the performance starts to fall off.

There was also another NEON related test, which was the Linpack Single Precision Benchmark. This related the speed to MFLOPS, the Pi 2 came in at 299.93 MFLOPS and the Pi 3 at 462.07 MFLOPS.

Worth it?

The Raspberry Pi 3 Model B is paving the way for not only a faster processor, but hopefully an inclination as to the future of the Raspberry Pi platform as a whole. The move over to supporting 64bit with the ARMv8 processor should allow the chip to be better supported by operating systems from now on. The addition of WiFi and Bluetooth on the board is welcomed by many, and this is one feature that I did not have time to benchmark. What I would love to do is compare the WiPi to the Broadcom on board WiFi to see what the throughput is like and to see if how the chip is now connected has improved the speeds at all. Though we may still be limited to the speed of connected storage (SDCard or even USB).

It would also have been nice to properly benchmark the OpenGL functionality of the VideoCore with the new open source driver and to compare that with the past iteration(s) of Raspberry Pi. Though it is likely there is negligable difference as shown by the comparisons between the benchmarks using Roy's OpenGL program (see attachment to this blog post), and it is nigh impossible to test is on the Raspberry Pi 1 range due to the low amount of supported RAM on the board.

Aside from the repeated call for "faster processor!" that people keep calling out, in some hope that a small board can replace their desktop computer, the Raspberry Pi 3 is still a gradual step in the right direction and the price of it is amazing. I find it highly commendable that the Pi Foundation have made everything backwards compatible so far, but this cannot last forever and the support for the systems will have to fracture somewhere as packages move over to 64bit, even if it does mean supporting them side by side. Time to educate the masses!

So, is it worth it?

Yes. Yes it is.

Bonus Coverage: Temperature

This was not by me, turns out the Raspberry Pi 3 runs hotter than previous models, enough that it should require a heatsink, and ventilation if contained. I discovered this was the case with the Raspberry Pi 2 overclocked to 1.1Ghz, so it makes sense. Though you will encounter people that claim the Raspberry Pi does not need cooling, I think we may see a change.

If I had a thermal camera, I would do a shot of everyRaspberry Pi. That would be awesome. Oh, SOMEONE DID!