Like the Zestimate’s results that are presented to the nearest dollar, this type of presentation infers a market precision that does not exist. Labeled as “disrupters,” I think Zillow’s concept of “aggressively marketing an accuracy that is non-existent” is one of the biggest challenges facing real estate appraisers and brokers and in turn, the consumers. Numbers can be magic, especially when presented in an econ-sounding way. The consumer absorbs the results as gospel without challenge.

I’m sure the numbers are accurate with the data they have but Zillow’s application of the results are misleading. Yes their data says blue bathrooms sell for $5,400 more but that doesn’t mean your blue bathroom will get you $5,400 more than your neighbor with the exact same house across the street that doesn’t have a blue bathroom.

Zillow has recently re-announced it is forecasting the value of each property out over the next year. It’s not a new tool for them, at least conceptually since the “What is a Zestimate Forecast?” page was last updated on October 3, 2012.

In a world with Big Data, it’s clearly inevitable to see an expansion of the capabilities of services from firms like Zillow and Trulia as their data set grows. Zillow’s Zestimate was a key web site feature at their launch (no listings!), but the company lit the real estate housing market industry on fire, establishing Zillow as a powerful brand that was here to stay, even if the Zestimate tool was problematic.

The challenges facing the Zillow Forecast tool

The Zestimates are still dependent on the quality of public record
Many markets (ie NYC), have quality-challenged public record. But as time passes, Zillow’s data set gets bigger and their logarithms get better and I have not doubt that the reliability will continue to improve.

If the Zestimate is wrong, the forecast will be wrong
Take a look at this chart on the highest price closed sale in Manhattan:

This is perhaps Manhattan’s most famous “trophy” sale of the past several years, 15 Central Park West. The property sold for $88M but the Zestimate at the time of sale indicated the value was $72M. However today the value is $11.9M and the forecast estimated an 8.6% increase next year to $12.9M.

The Zestimate Forecast projects the current Zestimate out over the next year using a bunch of indicators

I feel that most of these indicators, when considered as a group, are important to consider won’t capture the nuance of next year’s view because they either lag or aren’t granular enough to be a key influence on value trends over a short period. I would think Zillow would add search patterns and other “Internety” things to leverage their proprietary data to help with accuracy. I’d also consider “new inventory”, not just total inventory (supply) to help catch the nuances of a tight time frame of forecasting.

The key national factor driving nearly all housing markets now – credit – is really hard to quantify.

Still, forecasts are the future (sorry) and kudos to Zillow for taking the first step, even though the results, like the early days of the Zestimate, are probably not very accurate.

Comments Off on Zillow is Forecasting Future Property Values

Zillow has been one of the most visible and talked about AVMs (Automated Valuation Models) in the US and enjoyed considerable press during the housing boom. Of course they have always been at the mercy of the quality of public record data despite their technology prowess.

Perhaps they were more guilty of overhyping the reliability of their “Zestimates” in the early days by presenting value estimates precisely down to the dollar. But hey, it was cool to see how much your neighbor’s house was worth.

Zestimates on Zillow.com are no more accurate than homeowner’s estimates.

When it comes to using the Zillow.com automated valuation model (AVM) to get a free listing price on a house, users may be getting what they paid for, according to a report published by the Appraisal Institute that finds the Web site overestimates the values on homes almost as often as the actual homeowners.

Zillow has become the real estate punching bag to the real estate community. And once again, they are on the defensive in the media coverage of this report.

Here’s the issue:

The key issue regarding Zillow’s Zestimates is whether they reflect transaction prices. Zillow has been described both as “a useful site” and as “categorically wrong.” There have been many instances of praise and many instances of complaints by homeowners using the Web site to estimate the value of their homes. Realtors in general have also been critical of the values produced by Zillow.

Agents had issues with over valuation because they tended to set seller’s expectations too high. Of course, appraisers have an ax to grind with a service that was perceived to trivialize their expertise in valuation.

The report, “Zillow’s Estimates of Single-Family Housing Values,” was authored by Daniel Hollas, Ronald Rutherford and Thomas Thomson, doctors in economics, real estate and business, respectively. The report was published in the quarterly technical and academic publication of the Appraisal Institute, the nation’s largest association of real estate appraisers.

Online real estate’s posterchild for pricing inaccuracy (warning: I have been overusing posterchild lately) has been working hard to combat that image with the release of their 4th quarter market report. Zillow has been the subject of scrutiny for the wide range of inaccuracies in their pricing of specific properties called Zestimates (and their incompatibility with Mac’s Safari browser – must use Firefox). But hey, its still in beta a year after launch.

But market reports are a different animal. The attempt to value amenity differences (ie sq ft, room count, lot size, etc.) confuses many consumers and has been controversial due to inaccuracy issues. They buy national data feeds, not unlike many online service providers, and use this to come up with their Zestimates. The use of this data to attributes of a specific property is what has been tough for them (or anyone) to figure out to date.

Zillow has used this same data to crunch numbers for the quarter. However, their market studies use pricing as the main data point and there is limited consideration of amenity differences (other than location). This simplicity allows the reports to appear to be accurate, although I can’t vouch for every market.

When you consider the options for national data sources like NAR, which is a trade group and OFHEO which includes refi data in their results, you don’t have a lot of pure choices. That sounds like a cheap shot to Zillow but its not meant to be.

The report is a whole lotta spreadsheets by MSA, broken down by median sales price (Zindex). In the markets I am familiar with, the results seem to make Zense.

Zillow has been the punching bag of many of late, and perhaps its simply backlash from the large amount of free publicity they got after launch. It generated a lot of buzz to the point where bloggers were so sick of talking about them that they swore to abstinence (from Zillow) for a while.

People have had a chance to use the site for a while now and are beginning to realize how inconsistent the results are. Very accurate in some markets, very inaccurate in other markets and a lot of markets in between so you don’t quite know when and where the results are accurate. The results can be like snowflakes: pretty to look at, but the results for each search can provide a random result. They publish the ratings but I am sure the consumer doesn’t refer to them very often. And seriously, 52% accuracy in NYC means the results should not be published, should it? I have commented on this before since I am in the valuation business.

Zillow is such a neat concept on paper, but in the end, all data goes into a black box and spits out a number. They over promised and should have moved slowly over the country as they felt comfortable in each market. The logarithms that create the “Zestimates” are proprietary and perhaps thats what drives the suspicion, combined with pretty significant inaccuracy ratings.

Despite what I think are best efforts on Zillow’s part, however, I suspect we are seeing the first of several attempts to shut them down or change their model, refuting the argument that since they disclose their inaccuracy, they can continue business as usual. Because of their high visibility, its likely to become a public relations coup for anyone that goes after them.

In a letter sent by the National Community Reinvestment Coalition to the Federal Trade Commission last Thursday, the group asserted that Zillow’s Web site misrepresented home values and placed residents in low-income neighborhoods “more at risk for discriminatory and predatory lending practices.”

I don’t really follow CRC’s logic for taking this action since all markets are subject to various degrees of inaccuracy but its going to be interesting to see what the FTC does. I have seen this from my own personal experiences from family members in different parts of the country who have used the site.

Complaints are filed against appraisers for inaccuracy too, but unless there is negligience or fraud commited, its simply an opinion. I know first hand that data collecting in my urban market is often very challenging, and more difficult than in many surrounding suburban markets. Besides record keeping issues, one overlooked reason for the inaccuracy of urban areas is the challenge of valuation in a “vertical” market. Condos and co-ops stacked on top of each other is very difficult to automate.

Whats also really interesting, is the fact that they have been marketing to consumers in order to bypass the real estate professional, but recently the orientation has changed to try to be broker-friendly. This attempt to placate the brokerage community backfired recently and the pr spokesman for the NYT article contradicted themselves by saying:

[UPDATE]A Zillow spokeswoman, Amy Bohutinsky, said the site’s valuations, which it calls Zestimates, were intended for consumers and had never been marketed to real estate professionals. The company sees the tool as a way to empower consumers who in the past would have to rely on a real estate agent to make an estimate based on the sales of comparable homes in a neighborhood.

Is Zillow’s market ignorance in certain areas bliss? Thats up to the FTC, apparently.

“We believe the relationship between Realtors and their clients is going to change in the future,” Frink said in response to critics in the room. “As opposed to Realtors being gatekeepers of this information, they become much more of experts on what this information means.”

His is right and its already happening. 75% of buyers and sellers start with the Internet first before contacting an agent. Think about it, why would the Zillow concept, warts and all, be so popular? Is it because Realtors are accurate? Thats really not the point at all. Consumers want to gather information and digest it before they enter the sales process. The problem with Zillow, is its not consistent in its results so it promises more than it can deliver. Since the results come from a secret black box of algorithms, its a tough sell to real estate professionals.

But Realtor.com was pretty annoyed with Zillow [Telegraph] and went on the attack (aka the low road) which may have gotten some laughs and attention, but doesn’t help their cause at all. It paints a picture of the old guard, trying to keep things the same, which is not what the trade group needs to do.

Allan Dalton, president of the competing Realtor.com site, bitterly criticized Zillow.com, likening the site’s trademark Zestimates of home values to a carnival weight and age guesser.

Realtor.com, which is sponsored by the National Association of Realtors, provides listings of homes for sale by ZIP code from Realtor databases.

“The whole notion of suggesting to people that they can find out what their home is worth without a Realtor offends me,” Dalton said.

You’ve got to admire Zillow’s attempt to bridge the gap between them and the Realtor community. They are simply providing a new tool, valuable or not, to a trade group that is seeing the world change around them. However, its going to be a tough road for Zillow despite the fact that they are one of the most popular real estate sites on the web. Because a large portion of their target advertising market is real estate agents and the parent trade organization sees Zillow threatening their relevancy and causing more market confusion, its not a lovefest and probably never will be.

Whats interesting about the whole polarization of Zillow’s popularity, is that Zillow does not provide listings, yet are viewed suspiciously by agents. The initial rollout with promises of complete market coverage by Zillow, resulting in significant inconsistencies in results, is what has got many agents upset.

Zillow has promised more than it can deliver, which I have commented on before. Some market results are amazingly accurate, while some are not even close. The problem for the reader is having some way to measure its pricing reliability before it can be taken as a serious tool. Thats why the Realtors should not feel threatened at all.

Those that I have spoken to at Zillow are a very nice, earnest bunch of people trying to do what no one else has been able to. However, at the end of the day, (Of course I am not without bias here since I provide property valuations for a living.) the black box approach to valuation will always be viewed with suspicion. I guess thats besides the point.

Although I never went to Stanford Grad School, you’ll probably be just as smart by visiting the major upgrade to Trulia, which is being launched this Monday, September 25th. (Ok, I went to high school with someone who went to Stanford and its quite clear I will never be that smart.) Seemingly everyone involved in Trulia went to Stanford, so perhaps thats why their logo still says its in beta – they are studying for finals?

I got a demo of the new version yesterday in my office by CEO Pete Flint and VP Sean Black and I was impressed. Since I was first approached by Trulia last year before their New York launch, I have provided aggregate market data for them since then because I was intrigued by their product. I still am. There are a lot of listing mashups out there, but maps are last year’s news – everyone has them now.

Their new idea is to integrate public record, listing and enhanced mapping features to help drive traffic to the broker listings. It doesn’t cost the brokers anything and with weakening market conditions, demand for free products like this will likely increase.

Zillow is a pricing tool to give the user the estimate for a specific property using public record sources. Zillow added a new feature this week which allows property owners to modify public record features of a property since its hit or miss whether public record will capture accurate amenity features. My house is missing 2-bedrooms and about 1,200 square feet (I know I put them somewhere), but the value of my house seems ok for some reason. I still have high hopes for Zillow but not untill they get more consistent coverage to be relied on. When thats perfected, I think that the Zestimates will take you to the general realm of market value but will need to be fine tuned by a real, hopefully live, person.

My nephew in Ohio relied on Zillow heavily recently in understanding pricing in their area and used it for a reality check when they negotiated a home purchase. However, my in-laws’ house in Michigan is a tract housing subdivision and is valued at less than 20% of what houses we know recently sold in the past year.

Zillow seems to be saying now that they are a starting point for pricing, not the replacement for a professional to interpret the results.

Trulia is a listing aggregator that combines broker listings, with mapping technology and public record infomation and sales in the area to allow the reader to narrow down their choices and get comfortable with what values are and what is available for sale. In other words, the reader is put to work to understand the market and therefore owns the result. A really interesting feature of the new upgrade is the data that shows the amount of reader traffic that visits a specific neighborhood in order to gauge the amount of consumer interest in it. In other words, what is “hot.”

These are new features provided by their press release:

Trulia.com Interactive Heatmaps dynamically search your city to see which neighborhoods and zip codes are hot and which not!Â Consumers can search user behavior trends, median sales prices, average listing prices and more for every major city and every neighborhood across the country.

Trulia.comÂ Comparison ToolÂ searches data on over 60 million homes so you don’t have to.Â It works like this: you find a home you like, Trulia.com finds comparable ‘for sale’ and ‘recently sold’ homes to help you understand how your house stacks up, you make better real estate decisions.

Trulia Neightborhood Spotlight helps you answer:Â is this neighborhood popular?Â How do average prices in this neighborhood compare to the city?Â What about the quality of schools and crime rates?Â Which homes have the most bathrooms!?Â All that information is here.

At a party recently, I had the chance to meet Richard Barton, the founder of Zillow and he mentioned he was starting up a real estate site. He was a nice, very low key guy who happen to be one of the founders of Expedia.com, which turned the travel industry on its ear. His new site, Zillow got everyone’s attention and no one knew what it was – until yesterday. Inman spent a lot of effort peaking our curiousity and I got a lot of calls from people in the industry asking what the heck it does.

Wednesday was launch day. I read four articles this morning about the site and got excited to check it out for myself when I got into work. The NY Observer article was especially good. In fact I read it on my Treo as I commuted in to work.

As far as the media coverage goes, I find it interesting that technical tools like this are often painted as spelling the end of full service brokerage services. I find this point hard to accept. I think that tools like Zillow and others are a natural evolution of technology and special services like this offer something that full service brokers cannot provide and really aren’t in business to provide. I think its kind of like the iPod. Apple builds them but third parties build all the add-on accessories.

The result of these tools is a more efficient market because of the additional flow of information. Its also raises the bar for full service brokers to have staff that are more fully informed about the market. There is opportunity to interpret information. Over this next year or so, the number of transactions is likely to drop and many brokers who have relied on being order takers will now have to actually market. Those that always marketed in boom times, should have nothing to worry about.

Since the Zillow involves valuation, and I am an appraiser, I was especially curious because its such a daunting effort to automate valuation on such a large scale. In fact, for the most part, the lending industry has been trying to do this for the past 5 years with limited success (If you base success on accuracy rather than simply pushing paper for the files to keep the regulators happy). A few months ago, a national lender told me that out of the 10 major automated valuation services (AVM’s), 8 were totally unreliable, 1 was marginal and 1 was pretty good. This lays the groundwork for my initial skepticism about Zillow, but I am open minded. I think it will evolve and will have more strength in certain markets than others depending on the data they are fed.

Well, apparently, the public relations juggernaut the emerged over the past few weeks with the build up, overwhelmed the site early in the day and as of 11:51pm tonight they are still of the air. I found their ZillowBlog which explained the problem and put a human spin on it. They should definitely link the blog to their home page to keep a dialog of their technical progress.

For those who were lucky enough to get access, the reviews were pretty good but basically mixed (after all this is a beta and there is a lot more data for them to tap into.) Of course the “red light theory” seems to apply here. Users will likely only remember the valuations that were not accurate and not those that were.

I anxiously await my turn. I am sure this is going to be fun. More to come.

Follow Jonathan

Newsletter Signup:

If you haven't already, sign up for
'Housing Notes' to receive weekly insights and research.

First Name

Last Name

Email Address

About Jonathan Miller

Jonathan Miller is President and CEO of Miller Samuel Inc., a real estate appraisal and consulting firm he co-founded in 1986. He is a state-certified real estate appraiser in New York and Connecticut, performing court testimony as an expert witness in various local, state and federal courts. He holds the Counselors of Real Estate (CRE) and Certified Relocation Professional (CRP) designations. He is an Appraiser “A” Member of the Real Estate Board of New York and a member of Relocation Appraisers and Consultants, Inc.Learn More...

Subscription Service Coming Soon

You'll be able to choose from an array of robust housing metrics compiled using research developed during the preparation of our market report series. Expanded significantly from prior offerings, use this resource to build charts and custom data tables or leverage your own information for more powerful research and presentations.

In the meantime, here is a small sample of the aggregated data we will provide.

“Jonathan Miller: Best online real estate expert.”–Money Magazine

“A Curbed reader goes all 'Jonathan Miller' on us.”–Curbed New York

“Jonathan...understands how to take numbers and explain them to people in a way that makes sense.”–Dottie Herman, President and CEO, Douglas Elliman

“"Between the Bricks" Columnist gives a 2016 Data Brick to Jonathan Miller for keeping the industry honest.”–Lois Weiss, New York Post

“Jonathan Miller is well-known for taking the pulse of Manhattan real estate.”–PBS Nightly Business Report

Matrix Blog

Phil Crawford of Voice of Appraisal asked me to cover for him while he took a well-earned vacation. While I don’t have his sweet, syrupy smooth radio voice, I can grow on you a little bit if you listen long… Read More