How we test air conditioners

The cold facts of air conditioner testing

Split-system air conditioners are complicated machines and it can be hard to tell which one is better than another. We review them to help make your purchasing decision as easy as possible. Here's how we get the data that matters.

We've been reviewing air conditioners of various sorts for decades now. CHOICE participates in relevant Australian Standards committees and regulatory forums so we can keep an eye on industry trends and air conditioner regulations. However, we no longer lab test air conditioners (see how we test below for why we changed our testing method and how we go about reviewing them).

Our split-system air conditioner reviews focus on inverter reverse-cycle models as these make up the majority of air conditioner sales. While we no longer put split-system air conditioners through laboratory tests we do watch for unusual and interesting models and will put these to lab tests if the need arises.

Our current method

Air conditioners sold in Australia must be tested to the Australian standard AS/NZS 3823.2 and registered in a government database at energyrating.gov.au. This is to make sure they meet the required Minimum Energy Performance Standards (MEPS) and qualify for a star rating label. Government authorities occasionally perform check-tests on appliances that are subject to MEPS, to ensure they meet their required and claimed performance levels.

We source data on the models directly from manufacturers and cross-check it against the government registration database. Scores are then calculated with a lot number-crunching, spreadsheets and careful data-checking – just the sort of thing CHOICE people like to do!

This helps us bring you information on more models than ever before, and makes it easy for you compare them.

We're unable to calculate ease of use, but have previously found most models are good or OK in this respect. Use our air conditioner buying guide for tips to choose a model with a good remote control, as the remote is a key factor in whether an air conditioner is easy to use.

Not all brands respond to our survey. Without full information on specifications and ongoing availability, we generally don't include these brands in our report.

Why we changed our testing method

Before we adopted our new testing method in 2013, time and money constraints meant CHOICE was only able to review a very small number of models – about 20 per year, even though there are a few hundred split-system inverter models on the market at any one time. We could also only cover only one size category at a time – typically small (3.5kW and below) or large (7kW and higher) models. So the number of models we could report on was only ever a very small fraction of the market.

On top of that, we often had trouble sourcing models for testing as we needed to test well ahead of the season, and manufacturers often did not have their new-season models available in time. Add in the fact that it cost us about $160,000 to buy and test those 20 models each year (as of 2013 - it would be even more now) and you can see why we looked for a better value proposition for our members.

One of the main reasons we focused our lab tests on inverter air conditioners was that when inverter models started appearing back in the early 2000s, we were concerned that standard tests might not have been measuring their efficiency in a way that reflected their actual performance in a consumer's home. Now, inverter technology is the norm, and after several years of testing inverter models, we no longer have any particular concerns about their typical performance. So we are comfortable that a "desk review" is appropriate.

A model scoring 90% is excellent, 70% is good and 50% is borderline. Note that scores for previously tested models can sometimes change from their original published values, as the increasing efficiency and performance of new models occasionally requires us to rescale our scoring method. When that happens, older, less efficient models tend to score a bit lower than they did in previous reviews, and sometimes a previously recommended model drops off the recommended list as its score falls below the required level. That doesn't mean it's a bad model; simply that better ones have come onto the market.

Cooling and heating efficiency

Cooling and heating efficiency scores reflect how much cooling or heating the unit can deliver, compared to the amount of input power (electricity) it uses. The best scoring models deliver more kWh of cooling or heating per kWh of input energy – the result being they cost less to run.

Air flow

Air flow is a measure of how much air the unit puts out on its lowest and highest fan speeds. The lowest fan speed should deliver a gentle breeze, while the highest fan speed should deliver a powerful blast for cooling or heating the room rapidly. A model with a good airflow score has a useful range of fan settings; some models don't (the lowest fan speed is too powerful or the highest speed too weak, or both) and get lower scores accordingly.

Scoring the different sizes

We score the small (up to 4kW), medium (4 to 6kW) and large (over 6kW) models each on their own scoring scales, because it's unfair to put them all on one scoring scale. A large 9kW model just doesn't achieve the same efficiency as a small 2.5kW model, and would score terribly in comparison if we used the same scoring scale for both. Likewise, large models have more powerful fans and achieve much higher airflows, so we can't directly compare them to the small or medium models in that respect either.

So while the method for scoring each size category is the same, the actual scale is different for each of the categories. You should only compare scores within each size category. However, we've put all sizes into a single review for your convenience. Use the filters in the review to narrow your search by size.

Running costs

We also calculate annual running costs; how much each model costs to deliver 3000kWh of full cooling and 3000kWh of heating per year when running at its maximum capacity, with the remainder of the year in standby mode (based on electricity costs of 30c/kWh). This is a fairer comparison than simply comparing the cost of running the unit for a set period of time. That said, the calculated running costs are given as a guide only – your actual running costs will depend on how much you use the air conditioner, your home, local climate and so on.

Testing air conditioners requires a very specific laboratory, with controlled indoor and outdoor temperature and humidity. While CHOICE does have high-quality laboratories, including thermal labs for testing fridges, we don't have a lab suited to air conditioner testing – it would be too expensive to construct and maintain.

So instead, when we need an air conditioner tested, we send it to a qualified external lab.