Over the last four years, California Energy Commission has rolled out its HERSII rating system, which was designed as a way to provide a score to homeowners and prospective buyers that compares the energy performance of a house to similar homes (without occupant behavior). The HERSII system is based on the same engine that drives California Title 24 Energy Code and was identified as a key part of the State’s approach to driving energy efficiency in existing buildings.

The goal of the HERSII system is to label all California homes in order to equate energy performance with building value at time of sale, and as a way for a homeowner to get third-party recommendations as to the most cost effective options to save energy.

HERSII has been a centerpiece of the CEC’s thinking about how to implement Assembly Bill 758 (AB758), which is legislation that gives the CEC regulatory authority over existing buildings. While not explicitly called out as the solution in the legislation, the CEC has laid out a plan of attack that uses the HERSII rating as the driver to eventually mandate improvements. While there is a desire by many to consider alternatives, the HERSII policy has yet to officially change.

Before we move forward with this strategy, it is critical that we evaluate all of our options and the data from early HERSII pilots. This is particularly true in light of phase three of AB758 which includes the potential of mandating ratings and cost effective energy efficiency improvements on existing buildings.

Potential Mandatory Approaches could include:

• Requiring disclosure of energy performance and the completion of the most cost‐effective energy efficiency upgrades at appropriate trigger points in the life of all buildings, such as transactions that result in change of ownership, occupancy, or financing; replacement of major equipment; or building remodeling or renovation.

It was brought to my attention, that while the AB758 Action Plan did not discuss HERSII issue, there was an on the record discussion on HERSII at the AB758 Fresno Workshop where Commissioner McAllister and also Bill Pennington of the CEC, at which time both expressed a willingness to at least revisit or maybe even "reinvent" the HERSII program. I would like to commend them for being willing to go on the record, but I will also implore them that we need real resolution and clarity on this issue and that waiting for the next AB758 Report to emerge and then another proceeding is prolonging uncertainty that is affecting the market.

HERSII sounds great and asset ratings and third-party raters make for really nice policy white papers, but when put into practice on existing buildings, the results speak for themselves ― and don't necessarily agree with the theory. The HERSII ratings system is extremely costly, has low consumer demand, and does not result in significant conversion to energy efficiency projects.

In an attempt to develop a one-size-fits-all solution to provide an asset rating for use at time of sale and in the appraisal process and provide an actionable workscope and savings projection all in one system, we have instead created a solution that does not work well for either use.

So the big question is, will the HERSII strategy work?

The good news is that we now have results back from our initial tests of the system. However, the unfortunate reality is that the test phase has demonstrated serious shortcomings in the current theory, which should at a minimum call the HERSII system into question and hopefully lead to thoughtful evaluation of the shortcomings that emerge to avoid costly missteps for the marketplace and government programs.

Based on results from HERSII tests over the last four years, the HERSII rating system does not appear to be catching on with homeowners, nor does it seems to be driving market transformation in terms of conversion to energy savings or discernible asset value improvements (though the latter is harder to know with the current dataset). Admittedly, some of our data is a little sparse, but that is a function of how hard it is to get real numbers out of these pilots.

Can you guess when the incentives ran out?

What can we learn from the Sonoma County HERSII Pilot?

Sonoma County ran the largest HERSII pilot in the state, paying a subsidy of $700 per HERSII rating, which resulted in 100 percent free audits and ratings to customers, and unsustainably high margins for raters.

In Sonoma, we see a clear jump in the number of HERSII ratings while this lucrative incentive was on the table. However, shortly after the massive subsidy went away, the number of ratings drop immediately back to virtually the same number of ratings as were occurring before the incentive program. It would appear that even after this very expensive attempt to seed the market, there is little to no actual consumer demand in the market for ratings and one is left to wonder what happened to all the HERSII raters who got trained in Sonoma for this brief pilot, now that there is no more work.

Clearly, California does not have the funds to pay $700 or more per rating out of public funds on every home in California, so the fact that Sonoma’s 100 percent rebate pilot seemingly did nothing to jumpstart actual consumer demand is very concerning, and means an attempt to roll this system out would force California homeowners to shell out for an expensive service they are not valuing.

The fact that the Sonoma pilot program did not result in any discernable market transformation, either in terms of sustainable businesses conducting ratings or demand for ratings by homeowners, should be of serious concern.

Do homeowners value HERSII ratings?

When we look at the adoption curve of HERSII ratings during the recovery act period, we find that ratings are highly correlated to rating incentive programs and appear to have little organic consumer demand.

In fact, as is apparent in the chart on the right, it appears that these incentive programs had little lasting impact on demand in the marketplace.

When you couple this with the emerging facts related to the propensity of the HERSII system to overpredict potential savings by as much as a factor of 3x (see: Ex Ante Tool Review Findings Disposition for Energy Upgrade California Custom (‘Advanced’) Measure Savings, 1 Mar 2013), one realizes that not only is this system not working in terms of driving customers to take action, but it is failing to do so while massively inflating the level of savings and ROI being sold to the customer.

The propensity of the HERSII system to project significantly higher savings than what is delivered means that if HERSII in its present form was applied as a mandate, there would be many millions of California homeowners who would see only a small fraction of the savings they were told to expect.

This fundamental issue is based on the CEC’s attempt to create a system that is based on code, and then apply it to operational predictions of savings. These two uses are nearly opposite in terms of how a model is constructed, and speaks to the need for systems designed for the specific purpose being asked - one size fits all is not cutting it (learn more about this issue here).

The CEC may have the legal authority to regulate ratings in California, and AB758 may give the Commission the legal basis to attempt to implement this approach, however it is highly unlikely that they have the political capital to make it stick once we start subjecting real people to these outcomes.

Asset ratings may in fact have a place, but they are really not compatible with delivering actionable workscopes in the real world, and the one size fits all approach has resulted in a system that really fits nobody very well. HERSII is an expensive and complex system to deliver ratings that have significant accuracy issues, don't work for industry, and consumers don't seem to value.

Is HERSII worth the cost?

Each HERSII rating costs at least $500 and often more, based on the fact that conducting a rating consumes half a day in the field and then more hours back in the office inputting data into the California Energy Commission (CEC) mandated software tool. There is little room for substantial economies of scale.

When you consider that we have upward of ten million homes in California, the cost of having a HERSII rating done for all California homes will require California homeowners to invest $5 billion dollars in ratings - and that is just for ratings, not actual energy efficiency.

Do HERSII Ratings turn into energy efficiency projects?

California home performance contractors have a very hard time converting HERSII third-party audits into viable leads that result in customers who are ready to make energy efficiency upgrades.

HERSII Ratings conducted in Sonoma were cross-referenced with Energy Upgrade California™ (EUC) projects and it was found that conversion from rating to energy efficiency retrofit was under 10 percent (this analysis was conducted prior to 100 percent completion of the program, and until it is redone more accurately should be considered a clear directional indicator). This is especially surprising given that in the Sonoma program, unlike elsewhere in the state, home performance contractors were allowed to provide the rebate to customers for their diagnostic test-in as long as the project included a HERSII rating upon completion. Given that contractor close rates are typically 30 percent or greater, their participation in the Sonoma pilot would have been expected to inflate the actual conversion rates when compared to HERSII on the open market by third-party raters.

At a 10 percent conversion rate, a $700 rating incentive translates into $7,000 of public funds in rating incentive per closed retrofit in addition to Utility incentives, and other Energy Upgrade California program spending. Clearly this was not a cost effective way to drive demand for retrofitting.

This trend is borne out in PG&E figures for its entire territory, which shows that of the 582 homeowners who obtained HERSII ratings, only 59 projects have been completed energy upgrades through Energy Upgrade California, which pencils out to a similar 10% conversion rate. This of course compared to conversion rates of audits to retrofits that are typical of integrated home performance contractors of over 30%.

Why is it so hard to get homeowners to act on their ratings?

The HERSII process is convoluted, complex, and expensive. From a homeowner perspective, it goes something link this:

First, a third-party rater does a HERSII rating for a homeowner who shells out something like $500 for the audit and report. If that customer is interested in doing work, they bring the rating report to a home performance contractor to get the upgrades implemented. However, here is where it starts to gets tricky and the real world collides with theory.

Contractors are ultimately responsible for a project’s outcome, including its energy performance, and cost. When presented with a third-party HERSII rating, a contractor must determine if the report recommendations match the conditions he/she finds in the home. This means the contractor always must go back to the customer and do what amounts to another audit of the building to confirm recomendations, code compliance, and pricing, but — because the customer has already paid for a rating audit — the contractor has to conduct this work for free.

On a very large percentage of these third-party HERSII ratings, the contractor will end up not agreeing with either the estimated price coded into the rating (which drives the Return on Investment and ranking in the software), or they will disagree with the solution being recommended once someone with construction experience visits the site and creates an actionable workscope.

This puts the homeowner in a bind. On one hand, they have the third-party rater sporting a CEC logo on their report telling them one thing, and a contractor telling them another thing, and far too often the homeowner becomes uncertain as to how to proceed and who to trust, so no work gets done. The third-party model ends up with very low conversion rates, and is generally not a profitable source of customers for home performance contractors.

How do we move forward from here?

It is time that we focus our energies in California on moving to a market that relies on actual performance, not complex regulatory structures translated into software and a numerical scales. The web of regulation we have created has many unintended consequences, and with the advent of smart meters and big data, we have the opportunity to move beyond regulatory proxies and instead harness private markets and sources of capital. We should embrace this change as a great success of the California system, not a failure. It is time for State policy to advance and lead the nation towards a smarter more efficient model to deliver the deep energy efficiency required if we are going to hit our climate, energy, and economic goals.

In light of the results from HERSII to date, the California Energy Commission strongly considers adopting simpler approaches to providing homeowners with ratings, that are substantially lower in cost to homeowners and do not pretend to be of investment grade accuracy. One such solution is the national DOE Home Energy Score. A very simple model can be input through an API from a variety of software tools, and is simple enough to be part of a home inspection, resulting in a 1 to 10 score for the homeowner.

These simpler systems work as a way for homeowners to gauge a home's performance but will not be confused with real energy audits capable of delivering solutions to homeowner that contractors can actually build and that will deliver reliable results. We should separate upfront simple asset ratings from the more complex needs of a diverse contractor marketplace who can deliver the level of quality auditing necessary to develop an actionable workscope and a real price. Instead of applying an expensive and inaccurate comprehensive HERSII rating on every house in the State, we instead would only do an investment grade audit when there is pathway to getting a project built.We need to do more than tweak the knobs on the current model. We need to be open to new ideas and substantial change in our approach. Solutions are out there, but we need to be willing to admit that many current ideas are not working, and minor changes will not be sufficient to reverse the trend. It is time for California to step back into the lead and help the country move towards a sustainable market for energy efficiency.This process should start today by admitting that the HERSII system needs to be reevaluated based on evidence and feedback from the market. It does not matter how much we have invested, we need to look at empirical results and change course. We cannot afford to keep marching forward with the same basic theories, while hoping for different results.

When SCEIP decided to use their ARRA money to rebate audits in Sonoma County we advised them of the potential negative effects. Audits should never be free, it implies that they have no value. A good audit can help consumers save energy just by the information provided.
The result of their action (SCEIP) was lots of people getting audits with no intention of doing work. Good for the HERS raters, bad for the contractors. When it was too late to turn back, we tried to adopt HERSII test-in data to build jobs. We soon found out that in most cases we needed to verify the conditions ourselves in order to build an effective solution, as the HERSII raters often used vintage tables instead of real data.
It looked good on paper, let the HERSII rater test the house and the contractors build the job. In reality we found that in most cases we could not build jobs from their findings. In almost every situation we had to test the house ourselves again to build effective solutions. If the person collecting the data in the field is not using it to make money they tend to guess if they don't know the real answer.
In general the HERSII process is very complicated and is based on inaccurate software that we are forced to use.
We see the value in having energy ratings on homes, but the HERSII experiment is not the solution.

Scott Golden

2/20/2014 11:17:13 pm

Hi Charles,

It sounds like the problem that you're running into, is not necessarily the program that's flawed, it could be the HERS rater. My company has been doing HERS ratings for a while now, and we've had very good relationships with the contractors we work with. We pride ourselves on our accuracy, and high attention to detail.

I won't dispute that the mandated software is less than desirable, but I understand other options may be available soon.

I'm sorry you've had bad experience with HERS II raters in the past, but please don't paint all raters with the same brush. Just like mechanics, there's some fantastic ones, and some not so fantastic ones. Once you find a rater that offers the level of detail you're looking for, i'm sure your tune will change.

Scott,
I think we were working with some of the best HERSII providers in Northern California. One of the concerns is who is the expert in the room. HERS raters can tell clients all kinds of things that they have read about in books. Some work, some do not. Most HERS rates have never installed anything they are recommending to the client.
If they make a bad choice there is not consequence. It can be confusing to the client as to who to trust if the recommendations are different.
As the contractor, who will loose money if the suggested item does not work, we have a much different position about the actual outcome of the findings from the audit. If we get it wrong, we loose money. If the HERSII person gets it wrong they will likely never know, as they have no consequence to their suggestions.
This makes us, as the contractor, very careful about the actual conditions of the house and often requires us to verify the findings even though a HERSII has done an audit.
In it's simplest form it's who is taking the risk, it's not the HERSII provider.
Once the HERS rater is on the hook financially my opinions may change. Until then I see it as a simple solution for the program designers and not very effective for contractors.

While I don't disagree that "it is time that we focus our energies in California on moving to a market that relies on actual performance, not complex regulatory structures translated into software and a numerical scales," I do think HERSII might have a chance with a third-party partnership/hybrid model, and offer this for thought:

Here in San Diego we've been offering HERS II Rating Rebates for quite some time and we've seen great partnerships build between contractors (who don't like to test and model) and raters (who do test and model, whether they like it or not) that participate in Energy Upgrade CA Home Upgrade. As you know, advanced path projects require modeling. When contractors come upon clients that are fit for advanced jobs, they call a rater and have them do the test-in assessment and build the model that initiates the utility rebate. The contractor and rater work together to ensure that the measures included in the model are agreed upon, eliminating the need for another test by the contractor.

After our rebates run out, we anticipate that these relationships will continue, with the rater either being paid directly by the homeowner, or (more likely) the contractor will build the rater's fee into their invoice to the homeowner. Whether or not this type of partnership is sustainable beyond the utility rebate program, with it's modeling requirements, remains to be seen.

Over the coming year we'll be testing your theory around DOE Home Energy Score and home inspectors. I'm not convinced the HES process is that much less burdensome than a HERS rating. I'm also not convinced that home inspectors will be interested in adding this service to their business models, but we'll see. If you'd like to join in the conversation around design of the inspector home energy score program, let's talk - we'd love your input!

It sounds like you are involved in administering the San Diego HERSII subsidy program and it would be great if you could provide some data to support your assessment.

Given that the HERSII system for the entire State of California averaged only 65 ratings per month (out of 13.4M homes) in 2013, I am not sure how you can argue that the "complex regulatory structure" is working.

I also see from the latest EUC report that in 2013 and 2014 to date SDG&E has completed only a total of 46 total Home Upgrade Projects, which at an average project cost of $7,300 equals a total project volume of $336,000 - not exactly a demonstration of a successful model or enough volume to keep more than one single crew contractor in business.

The hope in the Sonoma pilot was similar that seeding the market the HERSII system would lead to market transformation. Clearly that did not happen. Can you articulate what conditions in the San Diego market support a different outcome?

Given that this blog was only able to analyze the sparse data that was available, it would be great if you could help provide data on the San Diego program.

To start with, how many HERSII Ratings are being done each month in San Diego, and how much is the total cost of the program (HERSII incentive + admin), and what was the conversion rates of those audits to retrofits (and the amount in dollars of the Upgrades generated would be the most informative).

My point about Home Energy Score was really more of a demonstration that there are other solutions out there, but I also think that it is very clear it has a lower transaction cost - the data input requirements are basically 20% of a HERSII inspection, and you can generate a score without having to use EnergyPro, and through 3rd party software using an API.

Yes. Clearly both HERS2 and EUC programs (single family) have failed. Mass consumer frustration and a scandalous waste of public funds. Very frustrating for those of us who know how simple (and cheap) it is to fix homes. Will the CEC and the program designers and managers do some soul searching? Lets hope so.
The home energy analytics (based on smart meter data) offered by StopWaste in Alameda County seems to be a big step in the right direction toward cheap asset ratings for homes. It will be very interesting to compare an analytics data set to contractor (or rater) findings in the same homes.
But real transformation in the residential market probably won't start to happen until we get universal benchmarking, at least at time of sale.

Scott Golden

2/21/2014 11:16:11 am

I like the idea behind a HERS Rating, as an added value to a home. It doesn't necessarily have to directly translate into a signed contract. Of course this is coming from a raters point of view.

I live up in Canada, and work with The Energuy in California. I worked for 3 years with Energuy Canada, through Canada's ecoENERGY program, where they had over a million homes take part in the program, and get rebates for home retrofits. Recently, the Ontario government passed a bill stating that ALL houses that will be bought or sold, will need to have a "Energuide Rating" associated with the home. This is the Canadian equivalent to a HERS II Rating. This will help ensure that homeowners will know what they're getting, adding value to home, and to ensure that the homeowners will have no surprises when the move in.

I think if we look at the HERS II rating as an added value for a home, and not necessarily a launching point for an immediate upgrade.

I do however agree, that offering 100% rebate for the cost of the rating, is silly. I know as a homeowner myself, i would definitely get a free rating, whether I could really use it or not, just for the curiosity of what it is. I would be more serious about the procedure, and care more about the outcome, if I paid for it myself.

This is a difficult problem to solve. My perspective is a bit different as a Rater and a Contractor. I see real value in a HERS II audit.

When performed correctly this model is valuable information. The HERS score produced could be of value to an interested and educated market.

I would add the vintage table which cut both ways is a joke. I am thankful that they have been thrown away in the new code roll out slated for July of this year I believe. Using the table instead of testing is just plain wrong. On the other side of the coin I would find conditions that were worse than the vintage table and was forced to use the table rather than my verified results. For instance a duct test with well more than 40% leakage could only be called 35% as that was the ceiling for the vintage table and I could not go above it. Same for insulation and other issues. This was silly on both flips of the coin.

The fact that EnergyPro is not accurate I believe has little or nothing to do with code. What EnergyPro fails to do if factor in tuning based on occupancy behavior. If you have a customer that heats the house in the morning cools in the afternoon and heats it again at night (yes I have seen this) no program in the world can accurately predict this homes energy use. While we can see it in usage or by an interview it does not correlate in software nor can it be tuned in.

I think EnergyPro does a fair job of what it is intended to do. It takes a home in Mammoth Lakes, the Salton Sea and the coast in Malibu of different sizes, construction, mechanicals and brings them into a fair playing field where they can be compared. Its predicted savings are over and under in the real world. Case in point is how little the model gives for duct and air sealing and how much it gives for tankless hot water. I have never worked with RemRate or other established programs, I would suspect that they have short comings as well. Modeling is a tool not a silver bullet.

For the model to work under the EUC program in my opinion you have to be savvy enough to game the software. I am and I refused to participate in the program based on false savings and forced recommendations, red tape to chase a rebate which was not in the customers best interest.

I think moving forward a weatherization format should be adopted. Rebates for air sealing and duct sealing with verified results. Attic insulation and wall insulation. Force each project with every customers favorite mechanical ventilation for IAQ.

To me the real issue is that for whatever reason the average home owner is not educated or interested in home performance and has no desire to be so. Once we shift this then the pendulum will swing. I think you point out that without free service the service is not bought. This is a shame and we need to work collectively to change this. I maintain it has value.