Category Archives: Benchmarks

First of all
No, I am not dead. I just don’t like to say stuff that no body want to read anyway so I kept my mouth (keyboard?) shut for a little while.

Second
Next review will be about whatever thing I’ll buy next. It will probably be an iPad or the new rumoured MacBook Pros. I can’t promise anything about this though.

Third, the reason of my post…
As many of you probably heard of already, iPhone OS 4 is coming soon and some of us, like me, already have the beta installed on their phones. Seriously, it is the best operating system I even seen on a phone since smartphone are in the market. There’s a lot of stuff in it that apple didn’t told you which can theoretically bring iPhone OS 4.0 up to par with any DESKTOP operating system out there. That’s just how great it is.

Unfortunately, all that we seem to ear about it these days are bad stuff… which is kind-off strange actually. A big majority wanted multi-tasking. Now that they’ve got it, all that we can read is stuff like: “I don’t really care about it, I would have preferred Flash that this.” Yes, this article is dedicated to you, people that doesn’t have any sense of mobile device logic.

Flash, a plague to avoid
The first reason why anybody would want to avoid Flash like a plague is about stability issue. Here’s a nice one, while I’ve been writing this article, look what popped on my screen!

See, it managed to crash while doing nothing! That little thing made me wonder so I started to investigate on was could be the reason of this crash. After running a little software called Shark, a performance analyzer tool that come with Xcode, I discovered somethings that should have remained buried. What it does it that is detect cache miss (moments when requested data is not available in cache and need to be pulled from an other cache level) in the L2 processor cache and list them.

I have exported the report to a text file and I’m providing it for you to read on this link. It is very easy to understand. The numbers to the left are the numbers of call to a function, the gibberish in the middle is the memory emplacement of the function and the text at the right is the name of the library that provided the function. Now keep in mind that this report was requested during a normal writing session. I have music playing in iTunes and a couple of software is running. Notice anything strange here? First process in the list is the mack kernel (or the OS X subsystems if you prefer). It represent roughly 14.2% of the entire cache misses which is normal. The second one though… I expected it to be iTunes since it is doing realtime audio playback and require frequent access to the processor cache but… no…

In fact, Flash Player represent 51.7% of all the cache miss that happened during this session. Basically, it means that half of the calls that the required my computer to access the RAM during the 10 seconds of profiling happened because of the Adobe Flash library. This is a very good reason to consider it like a plague as it use as much CPU time as a virus! In fact, flash player used 1 minutes 32.24 seconds of CPU time in 4 hours of not using it. As a little comparison note, iTunes only caused 4.4% of the misses and used 9 minutes 23 seconds of CPU time in constant playback during the last 4 hours.

So, what does it mean? It means that Flash is not only very hungry CPU wise but also memory wise. It also show us that it is not optimized at all and keep on doing more and more request to the system memory.

Touch is the future
Based on what we can see today, people consider touch screen devices as the way of the future. It does have a little futuristic sense to it but believe me there’s nothing impressive in them as they were in use since a good 10 to 20 years! Why am I talking about this? Because Flash is the complete opposite! Flash applications are generally built with a point-and-click interface so around 90% of them doesn’t work at all on a touch based device like the iPhone. There’s no way to access the various menus of the interface because you need to point at them and not click or tap on them. As a result even if you could run Flash on a touch based device like an iPhone, you would not be able of using it as it was not designed for this kind of usage.

Some websites like LinkedIn doesn’t use Flash and still have some compatibility issues with a touch based interface because they try to limit their content to the size of the screen and use scrollbars to let you access the rest of it instead of thrusting the web browser and letting it do it’s job.

Flash on the iPhone?
No way! Never! Not because it sucks performance wise and would drain your battery faster than light, but for the same reason I don’t want an HP Slate. It was not designed to be used with touch in the first place.

Advertisements

Rate this:

SSD Benchmarks
I have a surprise for you. Ho well… It’s not really a surprise since I’ve put that in the title and google analytics told me that there is about 91.89% of chances that you got here by googling “Apple SSD Benchmark”. Anyway, I’ve run a couples of test and the results are just astonishing. Here they are:

And that’s it, they’re already gone! That’s just how fast it is. I think the only way it could get faster is to boot directly from RAM! (Ho! That’s giving me an idea…) No seriously here is some figures:

Boot time is just under 17 seconds on OS X 10.6.2 and under 25 for Windows 7. This is blazing fast. But that’s just uninteresting suff. You see, when you boot your computer, you can go get a coffee or anything while it work. So this is hardly a time saver. On the other side, that next one is going to be of big help.

Safari‘s startup time is so small that you don’t even care. In fact, I couldn’t time it by hand. So, I went lazy and just didn’t done it. You’ll have to do with some words. There is simply no differences between opening Safari 4 and opening a new tab. That’s how fast it is.

Photoshop CS 4, on the other side, is a bit slow. You have to count up to 4 before you can work in it. In fact, this is the slowest application that I’ve tested. There is just nothing slower than that. I think it pretty much guarantee a 4 seconds or less for just about anything.

Conclusion
If you can, get one… don’t. As fast as it look, the Apple SSD cost a lot and has not much to offer. It doesn’t support TRIM nor does it have a garbage collector. That means that it will keep on slowing as you use it down to half it’s original speed. That’s very bad for a $ 600 option (MacBook Pro 13″) or even a $ 200 one (MacBook Pro 17″).

By now, you’ve also realized that there is no way to get one in the Mac Pro without getting it out from a MacBook which mean it is now a $ 2000 upgrade. This is extremely costly, specially when you can get the Vertex for around $ 350.

That’s it! See you next week for some other great news.

Rate this:

And it’s bad new
I have three very big project going on at school and the only thing I want to do next is kill myself. The bad new is not that I’ll die, it’s that I will not be able to write on my blog! I already switched to a 1-time-per-2-week schedule but it’s still not enough. So, don’t be surprised if you don’t hear about me for a while.

On the bright side
Do you remember, about 4 months ago I did some benchmarks of the samsung in my MacBook Pro? If you don’t, take a look here as it is like seeing the god of awesomeness. Anyway, here’s the big new: In that post, I said that I’d try to move the SSD in my Mac Pro to see how it stand aside the RAID. Well… It’s not going to happen today BUT… The SSD is now securely plugged in the Mac Pro with a handmade adapter built from the original Apple 320 GB HDD.

So as everything couldn’t get better, I’ll give you some early eyes-taken results that will make you lick your floor: safari open in half the time it was doing on the laptop and iTunes even faster. But wait! There’s more! My iTunes library is stored on my server which leverage the old RAID array and is connected using a gigabit link and it is STILL faster than the RAID alone!

Anyway, I’ll give some results when I’ll have some time to do the benchmarks. Now, I’ve go to go coding…

Rate this:

I’ve started a series of screen-cast on youtube that compare some basic features of Windows 7 and OS X Leopard from an end-user point of view. I try to keep a humoristic tone (specially when something goes wrong like in this first video :P). I do one take per OS unless something preventing me from publishing the video happen. The idea is to show how each OS react on a first try. The settings related to the tests where also rested to assure the authenticity of the video.

For the editing part, I try to keep it very close to the original. I will never cut a shot unless it’s preventing me from publishing the video. In that case, there will be a notice in the video. Everything is shot in HD to help you see what I’m doing during the screen-cast.

You can take a look on Youtube if you’re interested. The first part is about some basic file and screen sharing over different network type. Which one is easier to use and what kind of features are to be expected with the DEFAULT OS configuration.

Rate this:

Benchmarking of the MacBook Pro 15″
Last time we talked about virtual performances. Now, it’s time to talk about the real stuff. Let’s do some real world benchmarks! I’ll start with a basic boot time comparison between the Mac Pro, the MacBook Pro 13″ with standard HDD and the MacBook Pro 15″ with SSD. I’ll then test multiple applications startup at the same time and the lunch time of some major applications like Photoshop. The specs of both every computers are supposed to be on my blog.

MacBook Pro 15″
Apple SSD

MacBook 13″ Alu
5400 RPM HDD

Mac Pro 2008
4x Seagate 7200.11 HDDs

Boot time

Fastest

30.624 sec.

48.121 sec.

73.806 sec.

Slowest

30.989 sec.

52.506 sec.

82.113 sec.

Average of 3

30.825 sec.

50.07 sec.

77.467 sec.

Application startup

Safari 4.0.1

0.678 sec.

2.231 sec.

2.124 sec.

iPhoto 8.0.3

1.864 sec.

7.238 sec.

9.877 sec.

iTunes 8.2

1.315 sec.

7.192 sec.

5.322 sec.

iMovie 8.0.3

3.391 sec.

11.538 sec.

6.394 sec.

Photoshop 10.0.1

3.732 sec.

12.372 sec.

6.64 sec.

Pages 4.0.2

1.023 sec.

9.715 sec.

3.25 sec.

All of the above
(Simultaneous)

6.238 sec.

64.245 sec.

24.116 sec.

Analysis
Wow! I’d like to add that the boot time is the physical boot time. It combine EFI and OS X boot time. The system was considered booted when the desktop, dock and menu bar was visible and accepting interactions. The time for the EFI to startup is about 12 seconds. It should remain the same for whatever MacBook Pro you have. On the other side, OS X 10.5.7 did boot in 18 seconds which is blazing fast! The Mac Pro EFI needed about 40 seconds to power up since the RAID card start evey disk one after the other to reduce power stress on the power supply. Even with this 28 seconds gap, the MacBook Pro surpassed the Mac Pro by 18.642 seconds.

An other interesting fact is that it take half the time to start every applications at the same time compared to starting them individually. This fact point that the mass storage device (Apple SSD in this case) is not the bottle-neck of this computer. The fact that it is exactly half the time shows us that the CPU is the one slowing everything. When I started multiple applications at once, I took advantage of it’s multi-core architecture thus executing 2x more information at the same time.

Conclusion
The SSD option is definitely worth it. If you want a fast, silent and tough computer, it’s the first thing to get. I can’t imagine what one of those SSDs can do in the Mac Pro. It would surely be a screamer! Let’s say that it’s defenitly in my plan to do so. I will probably post a follow up when it will happen. In the mean time, let’s wait for the videos of those benchmarks and the review of my new iPhone 3G S. See you next week.

Update 30 June 2009 11:52
Those benchmarks might appear incomplete to you so, if you are interested in a specific value that is not present in the list, just ask using whatever means of communication you’d prefer. I’m also open to suggestions about hardware comparison.

Rate this:

Comming back
I finaly get the time to write this review. For those how might want to know, all my exams went very well and I have only one session left to do. Anyway, enough small talk and let’s get started! I’m sure can’t can’t stand the wait.

CPU for Pro needs
If it’s games, converting a bunch of video files or running VMware, you’ll find the T9550 very responsive. It should not be a very big bottle neck in that system. I often run many softwares at the same time (including Windows in a VM) and it generally don’t go over 80%.

I would not recommend the 2.4 in anycase. There is good chances that you’ll find the 2.4 MacBook Pro barelly enough for what you need. After all, we are in the Pro line here. Those are designed to do hard work. If you want a MacBook, the 2.4 model is probably the one that will give you the most for your money.

I’d say that the 2.93 gHz might be a bit overkill for this kind of computer. Even if you’re working with videos, You will proably lack of memory way before you’ll need a bigger processor.

Forgetting the lack of memory
The basic configuration come with 2 GB of blazing fast DDR3 memory which is nice but barelly enough to see how OS X can work at peek efficiency. If you get the small model, at least, take the 4 GB of memory option; Specialy if you plan on running VMs on your computer. On the other side, if you are very… hardcore at multi-tasking, you’ll want 8 GB of ram.

Many people will stop at the Apple website and take the 17″ just for that. WRONG! Never, absolutely never trust Apple specs! The 2.4 gHz is limitted to 4 GB of memory but the 2.66 does support 8 GB. You will find the memory kit on third parties websites and it generally cost 640 CAD. See? Always double check; Never trust Apple specs.

SSD for Super Storage Device
The MacBook Pro come with a 250 GB or 320 GB depending of your configuration choise. But as every Mac, there is a way to suck up evey bit of power you can get from the machine and replace it by an SSD. If the majority of people is still not convinced by SSD’s life time or prices, I am. It’s this kind of thing you don’t know how you manage to live whitout it before have one in your hands.

Apple use Samsung SSD’s which are in the world fastest and more robust ones. You have the choice between two options: the 128 GB and the 256 GB. For the first one, even if it look small, you will find it more than enough as long as you don’t intend to use boot camp. If you do, you should take the 256 GB one.

There is an other reason why you should try to priorise the 256 GB SSD. Here’s a rumor many people have heard: SSD do not slow down even if they are full. That’s not true. Well, at least, not completely true. Yes, read speed will never go down even if there is 1 byte of free space lest on the drive. Write speed, on the contrary will be affected. To prevent SSD’s from fast degradation, every sector is written on a random place in the drive. The more used sector there is, the more time it will take to find a free spot. SSD’s seek time are less than a tenth of a mili-second when empty but this process can make it go as slow as an HDD when full.

The best balance of used/free space would be about 1:1. That means that if you care about your write speed, you’ll need to keep about 55 GB free on the 128 GB SSD and 110 GB free on the 256 GB model. If you need more, you should check on what is really important and but the rest of you data on an external hard drive.

Two graphic cards?
Yes, many people think the second graphic processor is optional but it’s not the case. NVidia developped a technology called hybrid SLI which was designed to boost battery life on laptop that use powerfull GPU. It work by switching from one GPU to an other when needed. In the MacBook Pro’s case, it can give you 2 hours more of battery when doing casual work. Unfortunatly, because of software limitations in OS X, you need to do the change manually and to logoff after that. But rest assured, the 9400M is strong enough to handle most of your HD video playback and small games. Since I bought it, I didn’t needed to switch to the 9600M more than two times (one to be sure that it work, a second one to do the benchmakrs).

Our config
Do you like specs? Everybody like specs! Let’s have specs!

CPU

Intel T9550

Frequency

2.66 gHz

FSB

1066 mHz

Cache

6 MB

Memory

4 GB

Storage

128 GB Solid State Drive

Graphics (1)

NVidia 9400M with 256 MB of shared system memory

Graphics (2)

NVidia 9600M with 512 MB of dedicated GDDR3 memory

Benchmarks
Here come a little surprise. As I didn’t have any old MacBook Pro close to me, I had to find a replacement. I knew that this MacBook Pro was a killer but… I have to say… I didn’t even see that one coming. Here we are, MacBook Pro vs Mac Pro. Yes, you didn’t hallucinated, I did say that I will compare this little laptop to a monster and I have to say, they fit pretty well with the other.

Here’s a resume of the results with a little explanation from myself. I will take the result from the 9400M since they are more accurate.

Component

MacBook Pro

Mac Pro

Details

CPU

179.47

188.34

This one is a bit tricky. It doesn’t test the real computational power of the processor. It test how it handle it in a single threaded environment. It gives a pretty good way to show how architecture difference can influence the result.

Thread

317.13

831.36

This test look at how much power it can get from a multi-threaded environment. Here, you can see the big difference between the two computers. I never saw a benchmark that give valid result on the first shot. Here, the results should be much higher on the Mac Pro side since the xBench is limited to 4 thread.

Memory

186.12

202.00

The result is a direct comparison of DDR3 dual channel vs DDR2 quad channel. You can see that the DDR3 at 1066 gHz is a very good gain over the old MacBook Pro’s that used about the same memory as the Mac Pro but with only two channels. In fact, you can see that the new MacBook Pro’s memory is close to twice as fast.

Graphic suite

–

–

– Quartz

219.06

278.33

This test is about the same as a Direct2D test. It represent how the graphic card will render basic images, text and shape on the screen. The results are just as expected. The 8800 GT and the 9800 GT have the same overall performance so there’s nothing strange at seeing a 9400M GT under a 8000 GT.

– Open GL

174.20

181.60

This is an other strange result. How come a little 9400M GT is running as fast as a 8800 GT. I really have no clue. There is no reason for this result to be so close on both sides.

Storage suite

–

–

Ah! The interesting part.

– Sequential

73.80

275.81

As I say earlier, SSD’s benchmarks are very tricky to read. Here, we see that the Sequential read/write of the disk is way slower than the one from the Mac Pro. Here’s the little surprise. The Mac Pro use a Caldigit RAID card with 4 Seagate 7200.11 500 GB HDD in a RAID 5 setup. It can read an write about 270 MB/s but remember there is FOUR HDD here not just one. The details of this test show that in average, the MacBook Pro SSD is about the same speed as one Seagate 7200.11 HDD.

– Random

108.05

40.95

That’s where we can really the SSD shine. See? Not even 4 of the faster HDD’s in the world (actually in the top 10) can come close to one single SSD. The read and write speed are up to 16x faster than only one of the HDD.

Analysis
There is no way a simple laptop cpu could compare to the dual Xeon on the Mac Pro so what went wrong? In fact it seams xBench has a little trouble with newer computer. This little guy could not even peek both machine CPU’s for a tenth of a second. I don’t know how it test CPU speed but it’s probably not a good way and there might be some serious issues in the other tests too. So How do you have to interpret those results. Just say that everywhere you could do more if you do it more time simultaneously, the result will double on the Mac Pro’s side but should stay quite the same on the MacBook Pro.

Now let’s talk a bit more about the storage test suites. The Apple SSD is as fast as one of the best HDD in sequential access and is up to 16x faster than the same HDD in random access. An other important detail, the default 320 GB HDD in the MacBook Pro has about half the speed than the other HDD’s used in this test. That means that by adding a little $300 to your budget, you’ll get a 32x speed boost in your every day use. You should then choose this option over 8 GB of ram or a faster processor since it has a better value.

Conclusion
If you plan on getting a MacBook Pro, take the SSD. This is final. It’s not even a choice, it’s a necessity. Next week, I’ll add some other benchmarks in this review. They are real life example so they take more time to plan and process than any other benchmarks. Those tests will include a boot time comparison with the Mac Pro and it’s monstrous RAID array and a laptop that use a HDD from the same series as the standard MacBook Pro.

Update 30 June 2009 11:47
I finally posted some benchmark results. If you are interested in a specific value that is not present in the list, just ask using whatever means of communication you’d prefer. I’m also open to suggestions about hardware comparison.

See you next week

Rate this:

Intel got over its rival
I just got my hands on a very … let’s say surprising benchmark about the latest Xeon X7460 Nahelem processor. This CPU is in the same line as the Core i7 but are on the server side so we mean business here; and big ones. I will resume my analyze as this even if it might offense AMD fans here (just face it, it is called reality) :

Pick yours : Less than half the price or more than five times faster.

Indeed! Intel regain the first place in the PassMark benchmark with an outstanding score of 25 881 (as of 24 of march 2009) which is 1.9 times faster than the fastest AMD system available on the market who score only 13 600. What is really special here is that where AMD need’s 8 quad core processors to achieve this score, Intel only need 3. Long way to go AMD.

Update (25 of march 2009)
A bug seem to have appeard in the benchmarks and the score for the Intel X7460 was replaced by a AMD Sempron 1100 LE which should be in the Mid-End list.

Update (26 of march 2009)
The mark had been silently removed from the web site.

Update (1 of april 2009)
Apparently, the web site was updated and they removed all 2+ processors system from the list and moved them to a separated list named “multi-cpu system”. Still, the X7460 is not back on the list but X7350 (in a quad processor system) score 16 715 which is at least 3 000 better than the fastest AMD with half the number of CPU.