Last weeks "interesting" Gartner report on Windows reminded me I had some unfinished business. I've actually been waiting for Windows XP SP3 to release before I ran some tests, but I decided to go ahead and have some fun today while watching the Masters.
Round One - Cold Boot Timings
Round one goes to OS X and the Apple MacBook Pro. Surprised I said that? Well, it wouldn't be a very credible blog post if I lied, and frankly I wasn't really surprised at the result. The MacBook Pro is well known to boot, suspend and resume very efficiently and quickly. I haven't yet tested the suspend/resume timings. That will come later. One thing before we get to the details of the cold boot test, the MackBook Pro didn't spank the Lenovo ThinkPad T61p with Windows XP or Windows Vista.
I set each OS to boot and automatically login to an ID. As soon as the desktop was displayed and the browser icon was visible on an application bar, it was launched. Each of the browsers were set to hit my blog homepage. IE7 was the browser used for Windows XP and Vista. I tested both Firefox and Safari on OS X. The OS X Safari combo was the fastest combination. The MacBook Pro averaged right at 60 seconds for this test. But the ThinkPad T61p with Vista and IE7 was right behind is with an average of 1:10. Windows XP with IE7 followed that with 1:15. As you can see, there was no earth shattering difference. So the bottom line on cold starts is that if you are really worried about 15 seconds each day, you have too much time on your hands. Hit the power button and get your coffee. Each of the machines will be ready when you get back.
Round Two and Three - File Copies
Round two goes to the ThinkPad T61p. The second and third round of tests were file copies. I had heard so many stories that Windows XP totally kicks Windows Vistas butt, that I really wanted to see for myself. Unfortunately I waited until after I had already installed Vista SP1 so I don't have RTM test results. The SP1 test was interesting though. I have seen reports SP1 is actually slower than Vista RTM.
For the file copy test, I decided again to use something real world. I copied my documents around. I was careful to shutdown the machines and drives in order to flush any cache after each copy. The block of data (my documents) amounts to 7366 files, 1537 folders, clocking in at a whopping 42.7GB. Not huge, but enough to be annoying for file copies. The files are a healthy mix of big, medium and small files of all sorts.
I did some tests on the T61p that isn't possible on the MacBook Pro because the MBP doesn't have the ability to have two internal SATA drives. Not surprisingly, the T61p internal SATA to SATA file copy with Windows Vista SP1 x64 was the fastest copy. It accomplished the copy in a little over 19 minutes. As with the cold boot test, it wasn't an earth shattering difference in my opinion from the fastest file copy test to the slowest file copy test. You might note here that Windows XP Pro x86 took 30 minutes to complete the internal SATA to SATA copy. 11 minutes longer than Windows Vista x64 SP1. It's pretty clear to me from that test Vista is holding it's own nicely.
The next round of copies involved an external SATA enclosure using USB 2.0. Enter from stage left, the MacBook Pro. The MBP won this round, but it didn't best the 19 minute test above. I don't have a 34mm eSATA ExpressCard to use in my Mac, so I couldn't test using that interface. The first test I did was to copy the files from the external SATA drive to the internal MacBook Pro hard drive. I have the 160GB 7200rpm drive in my MBP. The MBP did that in short order taking only 22 minutes. From there, I did copies from the internal drives back out to the external SATA drive. I did that from Windows XP, Windows Vista and OS X. For the OS X test, I reformatted the drive with the native journaled OS X file system. OS X took 30 minutes. Windows XP took 36 minutes. Windows Vista took 40 minutes. As you can see, significant differences but hardly time enough to kill a cold beer.
Lenovo ThinkPad T61p with 4GB of memory, NVIDIA Quadro FX 570m, Intel T7500 Core 2 Duo. The MacBook Pro is nearly identical from a hardware standpoint. Same CPU, same GPU and the same amount of memory. The T61p has one thing that the Mac doesn't. The T61p has a multibay drive bay that allows my to insert a second hard drive and test SATA 150 to SATA 150 file copy performance. All of the drives used are 7200rpm SATA drives. The external SATA enclosure is a Vantec 3.5" eSATA enclosure and contains a 500GB Hitachi Deskstar SATA 300 drive. The USB 2.0 connection was used, not eSATA. eSATA could only have been tested with Windows because I have a 54mm ExpressCard which is incompatible with the Mac.
I learned something by doing these tests. The main thing I learned is that the performance of the three operating systems is closer than I thought. In fact, it's really a waste of time to debate it. In this busy age, a few minutes here and there aren't worth having the holy war I see waged when the various camps talk about the Mac and OS X, Windows XP and Windows Vista. Now granted we aren't testing applications here, and the tests above were really simple, but I'm comfortable with the results.
It used to be that day that you could point at the price of Apple hardware and software and question the premium they used to charge. Apple has smartly lowered those prices and removed that argument from the equation. Apple is also holding the Windows OEMs to a high standard by having a highly tuned platform. By owning both I can tell you there isn't a big difference in the performance and stability of the two platforms, if you are running hardware that is properly designed for Windows Vista, and the Vista installation is properly installed and configured.
The next few comparisons I plan to make will compare the ThinkPad to the MacBook Pro from a hardware perspective, then some of the features of the three operating systems. When we get to the user interfaces and features, we'll crank up the war of words. Who knows, you might be surprised what I write.
What about dos?
I've got a floppy disk here that will totally own you're silly graphical stuff when it comes to boot times
I think you are looking at it backwards. You should be after the slowest boot times (with the OS connected to "the enterprise"). :)
I think I will win. Today was about a 5 minute boot to see my XP SP2 desktop, then probably another 2 minutes before everything calms down so I can go into Outlook. Yes that is on my whopper Dell D810 3 year old machine.
Good point Steve, and I neglected to mention none of the machines are domain joined and require a directory server and authentication method of any kind.
Windows Boot times in the enterprise environment is quite slow. The go to lunch and come back to a slow computer syndrome is also annoying.
Brandon, like I said, "properly installed and configured" and for this series of tests, I kept authentication and directory services out of the picture. Most of the college students buying and using a Mac or Windows machine won't see a domain until after college so the point it really moot to them.
I personally hate my Dell Inspiron 1520. I think dell really screwed this machine up. It does not resume and go to sleep consistantly, the paint is wearing off, the lid creaks, I had to get a replacement a week after owning it...
Waking from sleep is very important to me, being a college student, I usually put my laptop to sleep change classes pull it out and resume note taking or work. But with this Dell, sometimes when I open my bookbag it is super hot in there because the machine never slept! I hate to shutdown completely because it does take a while to boot, plus I have to open up OneNote, Outlook, Windows Live Mail...etc.
I want to buy a Mac for this reason, so I can reliably sleep and wake, along with fast boot times. I just have to see if anyone wants to buy this piece of crap notebook from Dell.
I really feel done with Windows Machines. I liked IBM, my father had one but then the titainium hinge broke on it, and the motherboard died. Those laptops were so sturdy feeling, really like a tank, I just do not understand how it just broke. This was the T42 btw.
I'm glad you are exploring this...
College students do care about this, I do. I want to see some reliability tests done, just because. I wonder if it is Vista that causes my inconsistancies, or if it is Dell. I suspect Dell.
If it takes a long time to load into a domain environment, i dare say its set up wrong.
I had that issue once and come to find out, i had the DNS address typed in wrong so once i fixed that 5 minute logons went to a couple seconds.
There are a ton of laptops on the market. I have two Dell laptops. I have the Inspiron 6000 and the Latitude D820. Both are designed for Windows XP and run it pretty well. Neither run Windows Vista very well. My wife and daughter use them.
I also have two Lenovo ThinkPad T61p's. They run Windows Vista very well, and also suspend/resume quickly and efficiently. I rarely use suspend/resume because I am always pulling the primary drive and popping in my Windows Server 2008 or SUSE 10 drives. I plan to do some suspend resume tests over the next week or so to see if I spot any issues. Power management in the ThinkPad line is pretty good. I can't attest to any other machines from the other Windows OEM's.
I always wonder during these discussions how a consumer (or potential customer) of a particular laptop and OS is supposed to know if the darn thing really works. Most of the OEM's don't give you much time to use a device before the return policy expires. And there doesn't seem to be a good way to read quality reviews of long term use on the popular laptops.
From a student point of view, get a small fast machine that runs cool, has good battery life, and has decent power. Small and light is good in a backpack. Don't get a 17" boat anchor. The MacBook and MacBook Pro are small and light, but my Pro runs too hot. I don't like that. My T61p runs much cooler but it's 2 pounds heavier. But it screams, is rock solid, and runs Windows Vista very well.
If you are interested in another Dell, the XPS 1330 has received a lot of good reviews. That would probably be a solid choice.
I wish there was a try a laptop out thing on campus...
I need a pretty hefty notebook, which is why I got the inspiron 1520. It had the Nvidia 8600M GT and the santa rosa chipset. I run a lot of engineering apps, particularly those from Autodesk, like Inventor 2008...etc.
My friend has the Dell m1330, it is nice and all but the 8400M kinda makes me weary. Plus her plastic has warped a little bit, and that just bothers me.
I think I should have tried IBM again. But I do not know what their video card options are, it just has to be midrange. The only problem now is a laptop is such a huge investment. Buying another one less than a year later is just fiscally unsound. The way things are going I think I am going to be stuck with a laptop I hate for years to come.
My deepest desire is for Dell to go bankrupt. I am fed up with this crap. Just now when I opened my laptop, I got a bluescreen (not the typical kind) that said:
"*** Hardware Malfunction
Call your hardware vendor for support
NMI: Parity check/memory parity error
***The system has halted***"
I'm so ticked right now. I was doing a titration curve, to which I can't get an equation for to find the second derivitive to find the inflection point :(. If anyone can help I have a post about it
Anyways I'm ticked, I am going to contact dell today, they probably will do nothing, I will be switched between agents and nothing will happen. I used to love computers, but I hate them more and more, they cause headaches, people come to me to fix their problems, I have my own problems, I can't get stuff that works...
I have to disagree with your position that a 25% runtime difference on the copy test is significant (test wise) but insignificant (real life wise). In SMBs you often need to handle projects in a hands-on fashion, 50 machines x 1 beer. Can you say "worked the weekend"?
I am not discounting that your time is valuable. However, it the difference enough to warrant a platform switch or not?
Put another way, in my most recent tests, the T61p with Windows Vista was three times faster than the MacBook Pro at suspending and nearly as fast at resume. Three times faster sounds great, but the devil is in the details. Should someone worry about 30 seconds?
Certainly there's a line in the sand we all need to draw on time. If something is irritating enough, then by all means switch to something that is more productive. I do. If Windows Vista didn't meet my needs, I'd run Windows XP or something else. If Windows Server 2008 ran Sony Vegas, I'd be using it.
Trust me, I do these sorts of tests for several reasons. If Vista ends up being last place, so be it. The Windows team will get the feedback and have to address it. I think they are pretty focused on improving the performance and we are no where near done.
Thanks very much for the feedback.
I'm interested as to why you didn't install Windows on the Macbook Pro such that you'd have identical hardware for your tests. I'm interested to see what you find when XP SP3 comes out.
Good question Kabo. I'll answer both.
Regarding running Windows on a Mac, I have not setup bootcamp for a reason. If I want Windows on my Mac, I use VMWare Fusion. It's a better fit for my needs. That allows me to run all the operating systems I run in a more contained fashion via virtualization. Frankly I didn't think people would be that interested in how Vista performs on Bootcamp. I thought a more direct comparison between OS X and Vista woudl be more interesting to the readers of my blog.
Regarding XP SP3, it just became available so I wanted to wait for that. Unfortunately timing is bad for me. I am headed to MMS 2008 and will be unable to get to it for another 10 days or so. I'll probably run a couple of tests. XP SP3 on my T61p and on a five year old Compaq Evo n620c.
No promises though because I expect TechED 2008 to get in my way immediately after MMS 2008 so I will have to wait and see what happens.
With my Asus eee PC @ 900Mhz running a tweaked out version of XP SP3 I can be on the internet within 31 seconds. I wanna see a Mac running OSX touch that. If it can't, XP wins due to the fact it can be tweaked to boot much quicker.
31 seconds is pretty freaking fast. New winner. :)