Friday 5 February 2016

Benchmarking Guide

Benchmarking Guide

How well does your PC compare to your mates’ machines? Andrew Unsworth applies some new marks to his virtual bench

You’ve spent a small fortune building your dream PC and you know it’s ultra-fast, whether rendering videos in Adobe Premiere Pro or playing the latest blockbuster. You’ve overclocked it too, so the ordinarily fast PC you’ve built is even faster than the stock item. It’s better than your neighbour’s PC and faster than anything your mates have, but exactly how much better and faster is it? How can you back you up these claims?


If you want to compare the performance of one PC to another, or perhaps just one of its components, then you need to use benchmarking software. Benchmarking involves running a test on a PC or component that culminates in a result that that can be compared to others generated by running the same on a different machine, or under different conditions. A graphics benchmark, for example, will run a battery of rendering tests and then produce a result. If you run the same test on machines with different graphics cards then you can expect different results. However, you may want to test the performance of the same graphics card with different settings,  perhaps running one test at Full HD and another at Ultra HD resolutions. By using benchmarking tools, you can gain a set of verifiable results that’ll turn idle boasts down the pub into something concrete that you can use to get one up on whoever will listen.

Of course, there are many reasons to benchmark your PC other than curiosity and to find something to brag about. You may want to make sure that the processor you’ve bought is performing as well as the manufacturer says it should, or you may want to see if a graphics card driver update has had an effect on a PC’s gaming performance. If you’re shopping for a mobile phone then you may want to run a benchmark to test web performance while you’re in the shop, and we’ll show you how to do this later.

The Right Tool For The Job


As different benchmarking tools use different methods to test systems, it’s important to choose the benchmarking tool that most accurately tests the type of  performance you want to measure. The SunSpider benchmark will tell you how quickly a system and a web browser processes JavaScript code, for instance, which indicates how well a system performs when web browsing. It therefore doesn’t make sense to use this benchmark if you want to evaluate a system for video-editing work.

A benchmark suite such as PCMark would be a better indicator of video-editing ability. Alternatively, you could create your own benchmark by making a test project packed full of effects in your video-editing package, then time how long the file takes to render on the test machine and a control machine. If the control machine is a computer you already own and the test machine is a computer you plan to buy, then you can compare the results to see if the test machine has a performance advantage that makes it worth buying.

It’s also worth bearing in mind the effect that multiple cores have on the effectiveness of a CPU. If you run a benchmarking tool that only runs on a single CPU core then it might report that a Core i5-4690K is more powerful than a Core i7-5960X. That isn’t true, but this hypothetical result wouldn’t be an invalid result, either. The Core i7-5960X runs at a lower clock frequency than the Core i5-4690K, so if you’re running a single core benchmark then you’d expect i7-5960X to get a lower score than the i5-4690K. However, run a multicore benchmark and the supremely powerful i7-5960X would wipe the floor with the i5-4690K. Even so, the first (and still hypothetical) result is valid because you may only need to run single-core programs, and if you’re shopping for a CPU there’s no point spending over £750 on an i7-5960X when a Core i5-4690K will suffice.

It’s also worth bearing in mind that some components may operate differently and have different strengths and weaknesses compared to the competition. Take AMD’s Kaveri chips, such as the A10-7850K, for example. The Kaveri chips are optimised to compute OpenCL instructions very efficiently, so if you primarily work with OpenCL programs you’ll want to run a benchmark that tests OpenCL performance to get a more accurate impression of the A10-7850K. For similar reasons you should be careful to use the same driver when benchmarking graphics cards because the driver will have an effect on performance.

Once you start benchmarking, you’ll quickly find yourself tweaking settings in your operating system and BIOS to achieve even higher results. Once you compare your results to those of other people you’ll be hooked and benchmarking will become every bit as fun and addictive a hobby as gaming. In fact, I think it has the same effect on the brain as gaming because you’re presented with a challenge (trying to achieve a higher score in a benchmark) that you know you can beat if only you can come up with a strategy (overclocking or better cooling) to do so.

Indeed, overclocking goes hand-in-hand with benchmarking, with the latter being used to evaluate the effectiveness of the former and to test a system’s stability. Many benchmarking tools let you directly compare results with other users, prime examples being Intel Extreme Utility and 3DMark – to name just two.

As a last note, it may be an obvious point but you should keep your results stored in a database or spreadsheet. It’s possible that your benchmarking tool can store results, but if you uninstall it or it becomes corrupted then you’ll lose everything. It’s well worth keeping your results online in a service such as Dropbox or perhaps in a Google Docs spreadsheet.

Some of the best benchmarking tools we’ve used are detailed over the next few pages. Some are free and others require payment for full use. Take a look through them, choose the one that best suits your needs and get it used.

Mobile Benchmarks


Few branded smartphones are truly terrible these days, but if you’re paying £200 or more for a new phone you want to make sure you’re buying one with decent performance at the very least. This is especially important if you’re buying a phone with a custom operating system; if the phone on which your eye has settled makes special use of animations and fancy graphics you’ll want to make sure it has a decent graphics chip to push the pixels around. If you regularly make use of high-end apps for music production, video-editing, image-editing and so on, you’ll want to make sure it has a powerful CPU too.

Perhaps the most accessible benchmark for testing the performance of mobile devices is SunSpider 1.0.2, which tests a device’s ability to execute JavaScript commands and indicates web performance. It’s accessible because it runs in a web browser and doesn’t take long to run, so you’ll be able to run it while in a shop browsing for a new means of ignoring your fellow commuters. Just remember to note down the results.

You can access the SunSpider 1.0.2 benchmark at tinyurl.com/o39ql9g. Simply navigate to that webpage and click the Start Now! link in the Current Version section. The  benchmark will give you a whole host of results, but the most important one is the total time taken to complete the benchmark. Our Core i7-4960X-based test PC completed the SunSpider 1.0.2 benchmark in 162ms, while our iPhone 6 completed the test in 318.1ms.

You should note that the benchmark is affected by the web browser used to run the benchmark, so it’s possible to get a high completion time for a phone with one browser and get a much lower one with another browser. Our Core i7-4960X-based PC’s completion time of 162ms was achieved in Google Chrome. When we ran the SunSpider benchmark in Internet Explorer the completion time was halved to 80ms. Of course, both of these times are blisteringly quick. Similarly, our iPhone 6 achieved its 318.1ms completion time in the Safari web browser. When run in the Google Chrome browser the SunSpider benchmark completed in a much slower 1,224.7ms. For that reason it’s worth running the benchmark on the browser that you use most often to get a true reflection of how a phone or tablet will perform for you.

Pixelpushers


A good graphics processor is just as important in a mobile phone as it is in a PC, perhaps more so these days given that so many people use their phones and tablets to play games. My family definitely does. It’s therefore worth checking out FutureMark’s 3DMark app, which is available for for free on iOS and Android devices. 3DMark is a cross-platform benchmark, so you can compare a set of results recorded on one device to a set of results on another.

3DMark’s Ice Storm and Ice Storm Extreme benchmarks proved too easy for our iPhone 6 and it scored maximum marks in both tests. In the much more challenging Ice Storm Unlimited benchmark the iPhone 6 scored 17175, which is a great score, but not the highest score recorded for a mobile phone. Conveniently, 3DMark shows you scores from other devices on the results page so that you can compare your device’s performance to another. After running our benchmark, the app placed the iPhone 6 in third place, just after the Smartisan T1 in second place and LG Optimus Vu 3 in first place.

Our iPhone 6 3DMark result demonstrates a key purpose of benchmarking software, which is to see if your device or its components are behaving as they should. In the case of our iPhone 6, the result of 17175 is lower than the average iPhone 6 score of 17285, so our iPhone 6 is underperforming compared to the average. It’s only by a small amount and is nothing to worry about, but if the phone had severely underperformed it’d be a cause for concern and perhaps an indication of a defective component or other problem.

Of course, 3DMark is more than just a mobile device benchmark suite, and it has a long, illustrious history as a desktop graphics benchmark. There are three different versions of 3DMark (tinyurl.com/bkl86ct) for Windows-based computers: Professional Edition, 3DMark Advanced Edition and 3DMark Basic Edition. 3DMark Basic Edition is free, Advanced Edition is $25 (£16) and Professional Edition costs a whopping $995 (£638). The types of benchmarks and the resolutions and settings at which they can be run depends on the version you buy. The Professional version provides access to all the benchmarks, which can be run at default settings or at very specific settings.

The toughest benchmark is FireStrike, which is a DirectX 11 benchmark designed to stress the latest graphics cards with extremely challenging graphics and physics  tests. With a suitably powerful graphics card, users can even run a benchmark that renders video in Ultra HD. This is a tough task, but one that separates the Full HD also-rans from the truly high-end graphics cards. The other benchmarks are less computationally challenging and are suitable for benchmarking systems with mid-range graphics cards, general-purpose home or office PCs and gaming laptops. If you want to put your graphics card through its paces and see what it’s truly capable of then you should get hold of 3DMark.

Stress An Intel Till It Screams


This next benchmark is only for use with Intel’s K- and X-series processors, which are ‘unlocked’ and are able to be overclocked. The BIOS screen is the place where many overclockers apply their skills, but it’s also possible to overclock components from within Windows using tools and utilities. Many enthusiast-level motherboards and graphics cards come with such utilities, but Intel also supplies its own in the form of the Intel Extreme Tuning Utility (tinyurl.com/ptzcsvz).

You can use this utility to overclock their K- and X-series chips within Windows by manipulating sliders, and choose the clock frequency of each individual core, as well as CPU voltage. Users can also monitor CPU temperature, CPU utilisation, memory utilisation, CPU throttling, the number of active cores and the processor clock frequency to see how their overclocks have affected the system. Once tweakers have tweaked to their heart’s content they can try out the stress test, which absolutely clobbers the setup and is a good way of finding out if your system is stable. There’s also a stress test for the CPU and one for memory.

If your system is stable and you’re feeling brave you can run the benchmark, which uses the Prime95 (tinyurl.com/2atyo) application to derive a score that can be compared to others online at the click of a mouse. As Intel Extreme Tuning Utility is free and the benchmark is a great way of comparing your setup’s performance to those of other overclockers and enthusiasts, it’s well worth the download.

Desktop Benchmarks


PCMark is a high-end benchmarking suite and is made by Futuremark, the same company that makes 3DMark. As is implied by its name, PCMark is designed to test a system’s general performance, especially CPU performance, rather than just the graphics. It can test battery performance too.

A neat feature of the Professional version is the ability to use Microsoft and Adobe applications as part of the benchmarking process. This is important if you use such applications every day to earn money, or if you’re undertaking a review of PCs for potential deployment in your company and you need to know that it can do the job.

Every version of PCMark lets users benchmark their computers using a number of tests that simulate the workload expected of a computer. A computer that is intended for basic home use such as web browsing and basic productivity tasks is best served by running PCMark’s Home benchmark test, for example. The Home benchmark runs mock applications that tests a computer’s ability to browse the web, communicate by video-calling and edit photos. Two other benchmark tests, Creative and Work, use mock applications that tests a computer’s ability to perform media and office tasks respectively.

These mock applications run in real-time, so that it looks as if the user is performing these tasks, and the tasks are repeated three times. Other benchmarks include Storage, which tests a computer’s hard disk or SSD performance, and Applications – which, as already mentioned, requires the Professional version of PCMark in order to run.

It can take quite a while for the tests to cycle through, with the estimated time for the Storage test being between one and three hours. Once completed, though, PCMark lets users compare the results of the various tests online.

Like 3DMark, PCMark (tinyurl.com/kvlzer7) comes in various editions, with the free Basic Edition, the $50 (approximately £32.44) Advanced Edition intended for home use, and the $1495.00 (approximately £970.19) Professional Edition intended for governmental and commercial use. The latest version of the benchmark is PCMark 8, and it’s intended for use with Windows 7 and 8 machines.

Another good benchmarking suite is AIDA64 (www.aida64.com), which is a comprehensive testing and benchmarking suite. Indeed, AIDA64 provides in-depth information on many different aspects of your PC, such as information on the number of transistors in your CPU and the full timing and latency data of your PC’s RAM. The granularity afforded by AIDA64 is exciting and intimidating at the same time. It’ll take some time to get to grips with it, but persevere as it’s well worth it.

Aida64 provides disk, processor, memory (including cache memory) and graphics card benchmarks. Helpfully, you don’t have to run all the tests. You can, for instance,  just run a benchmark to determine your memory’s latency, or its copy or write speeds. Similarly, you can benchmark your processor with a single test. You could also, for example, determine the single precision floating point performance of your CPU using the Julia benchmark, which uses the Julia fractal set in its calculations to come up with a result.

Once you’ve run a benchmark you can compare the results to those of other systems, which will be a boon if you’re competitive or helpful if you’re worried your PC isn’t performing as well as you think it should and want to see how it performs against ‘lesser’ systems.

Some benchmarks can be run in one go as suites. An example is Aida64’s GPGPU benchmark suite, which runs a series of tests to determine your graphics card’s OpenCL performance. Each GPGPU benchmark can run on up to 16 graphics processing units (GPUs), and is compatible with Nvidia, Intel and AMD hardware. Examples of the individual tests include memory read, write and copy, and AES-256, which determines a GPU’s AES-256 encryption performance.

If you want to know absolutely everything about your PC, you should definitely check out AIDA 64. The latest version, 5.50, provides data on NZXT’s Kraken liquid-cooling systems, USB3.1 peripherals, Socket 1151 motherboards and Corsair, Logitech and Razer RGB LED keyboards among other things.

Aida64 is even available as a free trial, so you can see if it’ll meet your needs before you cough up the beans for it. There is also an iOS app that lets you view detailed information on your iOS device’s sensors, as well as other items such as the battery, memory and other important metrics.

People benchmark PCs and components for all sorts of different reasons, whether it be to evaluate a PC model when they are thinking about buying 1,000 of them for an enterprise, to see if any given phone has a more powerful graphics chip than another or to see if their latest overclocking strategy has squeezed even more power out of a processor than they thought it would. Whichever one you use, just remember to choose the correct one for the job, and tell us about your exploits in the letters pages.

Cross-platform Benchmarking


Benchmarks test the performance of a specific computer or one of its components. If you test the performance of a graphics card on one Windows-based PC it should give a similar performance on another Windows-based PC, assuming one of them isn’t of too old a vintage or otherwise compromised. What if you compare the performance of different devices with different operating systems, though? What if you can’t decide between an iPhone 6 and an LG G4 and want to see which has the best web or graphics performance? In that scenario you can turn to a crossplatform benchmarking tool to help you reach a decision.

Examples of cross-platform benchmarking tools include Webkit’s SunSpider and Futuremark’s 3DMark. To use SunSpider, for example, simply point your browser to tinyurl.com/4h87sa2, click the Start Now! link and record the result right there and then.

Run the same test on a different machine, record the result and compare the two results. Whichever device has scored the lowest result has the better JavaScript performance and in my experience has better web performance. Just remember that in the case of SunSpider the web browser used also has an effect on performance, so benchmark devices using a few different browsers.