A Call For Measurements: Isolating the Thunderbolt Effect.

I was thinking about this, and I think I came up with a way to test the effect of Thunderbolt 2 and/or Thunderbolt 3 on GPU performance. I have no means to carry out these measurements because I do not own a desktop, but I was wondering if someone would be willing to pick up this glove. We might need to raise a little money for this effort as it requires some hardware most people won't have, but I am sure the community can get together to figure this out.

Here is my proposal:

Requirements:

A modern, Windows-based gaming tower: Something like a i7-6700K, 16GB of RAM and a GTX1080, or another high-end gaming card.

A TB3 eGPU enclosure with updated firmware.

A PCIe to Thunderbolt3 add-on card. This is the part that people that own both an eGPU+Laptop and a gaming tower likely do not have. Alternatively, someone who has a desktop with a Thunderbolt3 port on the motherboard. These motherboards do exist, but they are very rare.

A 4K monitor (to test a wide selection of resolutions from 1080p, through 1440p/1600p and up to the full 2160p).

The idea:

Measure the same system, with the same CPU, the same memory, the same everything. The only variable will be the connection of the GPU to the system. In the first case, it will be connected directly to the PCIe x16 slot. In the second case, it will be placed in the eGPU enclosure that will connect to the TB3 add-on card placed in the same slot (a bonus measurement would be connecting the TB add-on card to a slot that goes to the PCH, as opposed to the CPU directly).

The end result should be as good of an isolation of the Thunderbolt3 performance reduction factor as we can hope for. Bonus points if the same can be repeated with a TB2 setup to see the difference between the two. This is better than comparing a desktop vs a laptop, because it keeps more variables the same, and it is better than comparing a dGPU with an eGPU because we remove potential CPU bottlenecks from the result (they are a valuable question by their own right, but that is not what I want to measure here).

I was thinking about this, and I think I came up with a way to test the effect of Thunderbolt 2 and/or Thunderbolt 3 on GPU performance. I have no means to carry out these measurements because I do not own a desktop, but I was wondering if someone would be willing to pick up this glove. We might need to raise a little money for this effort as it requires some hardware most people won't have, but I am sure the community can get together to figure this out.

Here is my proposal:

Requirements:

A modern, Windows-based gaming tower: Something like a i7-6700K, 16GB of RAM and a GTX1080, or another high-end gaming card.

A TB3 eGPU enclosure with updated firmware.

A PCIe to Thunderbolt3 add-on card. This is the part that people that own both an eGPU+Laptop and a gaming tower likely do not have.

The idea:

Measure the same system, with the same CPU, the same memory, the same everything. The only variable will be the connection of the GPU to the system. In the first case, it will be connected directly to the PCIe x16 slot. In the second case, it will be placed in the eGPU enclosure that will connect to the TB3 add-on card placed in the host.

The end result should be as good of an isolation of the Thunderbolt3 performance reduction factor as we can hope for. Bonus points if the same can be repeated with a TB2 setup to see the difference between the two. This is better than comparing a desktop vs a laptop, because it keeps more variables the same, and it is better than comparing a dGPU with an eGPU because we remove potential CPU bottlenecks from the result (they are a valuable question by their own right, but that is not what I want to measure here).

Thoughts?

i think that also a clevo p750dm2-g/p775dm3-g or an eurocom tornado f5/msi 16L13 with a cpu i7-6700k/i7-7700k and tb3 could make this test. or 6820hk/7820hk overclocked?

To do: Create my signature with system and expected eGPU configuration information to give context to my posts. I have no builds.

As long as the dGPU and the eGPU use the same exact chip at the same exact clocks (or can be over/under clocked to match), then this can also be done with one of the "desktop replacement" laptops, yup.

This is the same as @itsage done with Alienware R3 13, he added 1060 as eGPU to laptop with dGPU 1060 so maybe it is waste of time and money, my thread about PCI Express vs. eGPU is also the other ways to find how the performance drop is, its not accurate, but users must know that with External Display the performance drop is 20% for TB1, TB2 and TB3 and that the Internal Display performance drop is based of the GPU, the Thunderbolt number because the bandwidth goes in two directions not in one like with External Display (Monitor)....

I'am sure that the test you want will show the same results + or - 5% less or more

My reason of thinking about this is twofold: One is purely academic. I want to see this measured on a gaming desktop, where the CPU is of no issue. Sort of a fresh look at the same question.

The second is the notebookcheck review of the Core, where the GTX1070 (or higher) eGPU is barely of any effect over a dGPU GTX1060. That doesn't seem to line up with a 20% drop in performance, as the GTX1070 itself is more than 20% faster than the 1060, and the 1080 and 1080Ti are far, far more powerful.

In addition, with such a benchmarking setup, you can test ANY card to establish the bottleneck effect, whereas comparing a dGPU to an eGPU is limited exactly to the dGPU chip that you have. That in and of itself is valuable information.

I'd nominate benchmarks comparing TB3-CPU (15" MBP) architecture versus TB3-PCH (every other notebook). That's the last holdout that may explain TB3 underperformance. There we'd need back-to-back TB3 gaming benchmarks on two CPU+eGPU alike systems but one would be a 15" MBP. There we'd isolate the effect of the device shared PCH and possible DMI bottlenecking/latency on TB3-PCH Intel mobile architecture.

If TB3-CPU benchmarks are not significantly better than TB3-PCH then it would point to the TB3 controllers having performance issues. TB3-CPU architecture matches desktop systems' where the PCIe bus is attached on the CPU, example from here:

Skylake-S needs a new socket, new chipset and new motherboard. The socket is called LGA 1151 and has a single pin more than Haswell, the company’s current generation Core architecture. The new desktop processor supports both DDR4 1.2V and DDR3L 1.35V and it will be up to motherboard manufacturers to support one or the other.

The chipset that will let you overclock Skylake-S is called Z170 and it comes with 1 x 16 plus 2x8 or 1x8 plus 2x4 PCIe express 3.0 combination...

Performance and daily experience

For the purpose of my testing, I compared the results of my current desktop with the Razer Blade 2016 attached to the Core. Both used the same video card, an Nvidia GTX 970. The desktop has a Core i5-3570K (with stock clock speeds) processor and 16GB of RAM, while the Blade gets an Intel Core i7-6700HQ processor with 16 GB of RAM as well. I also tested with the XPS 15 which has an Nvidia GTX 960m paired with the i5-6300HQ.

Some time after I initially wrote this article, I was able to repeat the Razer Blade and XPS 15 testing using a reference GTX 1070 and then again with the Asus Strix GTX 1080. My desktop is dismantled now so I couldn’t repeat the tests for that one.

All benchmarks used a QNIX 2560 x 1440 px monitor, and the laptop’s display was disabled when attached to the Core. I was able to overclock my monitor to 96Hz pretty easily, so we could see actual framerates above 60(except for Fallout below native resolution).

Desktop w/970

Blade + Core+970

Blade + Core+1070

Blade + Core+1080

Blade

XPS 15 + Core+970

XPS 15 + Core+1070

XPS 15 + Core+1080

XPS 15

3DMark – Fire Strike

9168

8742

11554

12797

6513

7433*

8266*

11607

3561

3DMark – Sky Diver

21371

22692

26401

27315

18741

17746*

18599*

22909

9949

Fallout 4 – Outside the Corvega plant entrance with a battle

Desktop w/970

Blade + Core+970

Blade + Core+1070

Blade + Core+1080

Blade

XPS 15 + Core+970

XPS 15 + Core+1070

XPS 15 + Core+1080

XPS 15

Ultra, 2560 x 1440 px

43-53 fps

38-48 fps

48-63 fps

55-65 fps

N/A

32-41 fps*

44-60 fps*

49-60 fps

N/A

Ultra, 1920 x 1080 px

51-62 fps

52-60 fps

54-60+ fps

54-60+ fps

42-52 fps

43-60 fps*

48-60 fps*

54-60 fps

23-32

Witcher 3 – Walking around the grounds on the first tutorial

Desktop w/970

Blade + Core+970

Blade + Core+1070

Blade + Core+1080

Blade

XPS 15 + Core+970

XPS 15 + Core+1070

XPS 15 + Core+1080

XPS 15

Ultra, 2560 x 1440 px

34-38 fps

24-26 fps

34-38 fps

40-48 fps

N/A

20-22 fps*

33-36 fps*

40-48 fps

N/A

High, 2560 x 1440 px

42-47 fps

34-38 fps

48-50 fps

57-60 fps

N/A

29-31 fps*

48-50 fps*

57-60 fps

N/A

Ultra, 1920 x 1080 px

45-53 fps

28-31 fps

41-46 fps

48-54 fps

28-31 fps

22-25 fps*

42-46 fps*

46-56 fps

17-20

High, 1920 x 1080 px

60 fps

45-53 fps

60 fps

60 fps

39-41 fps

38-40 fps*

60 fps*

60 fps

23-25

Dragon Age Inquisition – Battle nearby a camp at the beginning

Desktop w/970

Blade + Core+970

Blade + Core+1070

Blade + Core+1080

Blade

XPS 15 + Core+970

XPS 15 + Core+1070

XPS 15 + Core+1080

XPS 15

Ultra, 2560 x 1440 px

32-35 fps

22-28 fps

32-40 fps

45-48 fps

N/A

23-28 fps*

22-28 fps*

43-48 fps

N/A

High, 2560 x 1440 px

53-61 fps

55-60 fps

57-64 fps

67-75 fps

N/A

42-47 fps*

40-48 fps*

63-73 fps

N/A

Ultra, 1920 x 1080 px

46-54 fps

35-42 fps

48-56 fps

58-63 fps

27-31 fps

35-40 fps*

35-42 fps*

52-64 fps

17-20

High, 1920 x 1080 px

75-89 fps

60-68 fps

73-90 fps

80-98 fps

48-52 fps

55-64 fps*

55-65 fps*

76-88 fps

26-33

Crysis 3 – Opening mission

Desktop w/970

Blade + Core+970

Blade + Core+1070

Blade + Core+1080

Blade

XPS 15 + Core+970

XPS 15 + Core+1070

XPS 15 + Core+1080

XPS 15

Very High, 2560 x 1440 px

31-45 fps

25-45 fps

35-65 fps

40-75 fps

N/A

25-40 fps*

20-50 fps*

40-65 fps

N/A

High, 2560 x 1440 px

43-65 fps

34-65 fps

49-96 fps

57-110 fps

N/A

35-60 fps*

33-75 fps*

57-95 fps

N/A

Very High, 1920 x 1080 px

46-67 fps

40-65 fps

46-85 fps

52-100 fps

29-42 fps

37-60 fps*

29-65 fps*

50-85 fps

17-27

High, 1920 x 1080 px

65-96 fps

53-85 fps

59-96 fps

71-120 fps

44-60 fps

46-82 fps*

45-80 fps*

65-103 fps

25-40

*readings on the XPS were taken with bios version 1.2, which was found to(sometimes) hinder performance of the GPU slightly. I can’t go back and remeasure them since I no longer have that GPU. The GTX 1080 readings were taken on bios 1.1.19, which is much more consistent.

I also added a few logs showing frequencies and temperatures of the Razer Core connected to the Razer Blade and my 1440p monitor.

As you can clearly see, the Razer Core certainly does a good job boosting the performance over the Razer Blade alone. If you have the new Nvidia GTX 1070 or 1080, it improves those graphics settings even more to get some really decent fps on AAA titles. Even the 970 shows some significant signs of improvement over the 970m, if you don’t want to spend that kind of money.

I’ve been doing a lot of research since originally writing this section and there have been a number of things I found out over the past few weeks. The first thing to mention is the performance hit the Core takes as opposed to using a desktop. If you compare my desktop+970 readings to the Razer Blade+Core+970, you’ll notice a significant fps drop. The CPUs in both machines offer very similar performance scores in benchmark tests, so the only thing I can chalk off the performance hit to is losses through the thunderbolt connections/driver issues. So if you’re planning on using a particular graphics card and are comparing it to desktop benchmarks, expect to see a 10-15% performance drop right off the bat.

Another thing I want to mention is the performance with the newest Nvidia GTX 10xx series cards. As you can see, my Firestrike scores are much higher than the 970, which is what you would expect. But if you compare those results with Firestrike benchmarks taken in people’s desktops with the 1070 or 1080, you’ll see a much larger performance drop(nearly 30%). On top of that, the Firestrike performance of the 1080 is only 10% higher than the 1070. At first I thought maybe the 1080 is being bottlenecked by the Core somehow. I ran another test though(which I’ll get to shortly) and determined it wasn’t the bandwidth limit being reached. My only conclusion is that the CPU in the Razer Blade is skewing the Firestrike scores to be really low. Here’s why.

The score is actually calculated off of the weighted average of three scores: Graphics(GPU only), Physics(CPU) and combined. Since the CPU is the same in all tests, the Physics score is unchanged. And since the Graphics score has such a large increase, but the CPU score stays the same, the average is going to be lower when compared to someone testing the card in a machine with a top of the line Skylake desktop CPU. In other words, a 20% increase in graphics performance equates to roughly a 10% increase in the Firestrike(standard) score. This would be a good place to use the Firestrike Extreme score as a comparison since it relies more on graphics performance at high resolutions.

The good news is this isn’t really a factor with many titles since they are more driven by the GPU than the CPU. You can definitely see a more proportional performance jump between the 1070 and the 1080, when looking at the fps measurements I got in the games I tested. It’s certainly better than the 970 readings and much much better than using the 970m in the Razer Blade. To the point, I’m happy with the performance of both the 1070 and 1080 in the Core.

So how did I determine that I wasn’t hitting the bandwidth limit of the Core? Well since I got the Asus Strix 1080, I decided to overclock it and see if I could improve things. My original Firestrike score was 12797(Graphics: 16764, Physics: 9603). I overclokced the GPU boost clock to 1973 Mhz, the memory to 11.2Ghz and let the voltage alone. My Firestrike score jumped to 13294(Graphics: 17785 Physics: 9610), a 6% gain from a ~10% overclock – not bad! I didn’t go crazy testing games with the overclock but I did gain 3-4 fps on the Witcher on Ultra QHD settings.

In regards to the XPS 15, you’ll notice my benchmarks are a little all over the place between GPUs. It’s because the system is a little sensitive to a number of factors(which I describe in more detail below). In short, if you use the wrong bios your performance will differ. For the 970 and 1070 tests I was on bios 1.2 but for the 1080 I was on bios version 1.1.19. The main thing it allows is for more consistently higher GPU usage, whereas before it was erratic and sometimes throttled for whatever reason. I left the XPS 15 + 970 and 1070 benchmarks on the table but take them with a grain of salt.

You might have noticed that even with the 1080 and the better bios, the XPS 15 still has a lower Firestrike score and some of the gaming benchmarks are a little lower than with the Razer Blade. I looked into it and it’s definitely because my XPS 15 is the i5 version. I’ve been collaborating with Doug who also recently got a Core. He has the i7 version of the XPS 15 and got a Firestrike score more similar to what I got with the Razer Blade. The physics portion of his score is 9973, which is more consistent with the Razer Blade. The physics portion of my score was only 6165. So if you’re looking for the bext performmance and want an XPS 15, aim for the i7 version. It’s still a buggy connection though, which I describe more in the section below.

Would this ASUS ThunderboltEX3 card work on an Alienware Area 51 R2? If so, I can get this project underway. I read the AW Area 51 R2 uses a custom X99 motherboard. At one point, the BIOS even had Thunderbolt support but it was beta and later removed. Here's the specs of this PC: