I think I already made it clear that I’m no fan of automated home valuation models and Zillow’s Zestimate in particular. Apparently quite a few other people aren’t either. Just last week I posted an article on how some Chicago area builders and their attorney (also a home seller that feels burned by her Zestimate) have filed a class action lawsuit against Zillow.
Although the lawsuit takes issue with Zillow’s right to even create, much less post, their Zestimate clearly this would be much less of an issue if the darn thing was accurate. Zillow knows this so they spend a lot of energy on improving the accuracy of the model and then bragging about how great it is. Just last week when this class action lawsuit was reported (coincidence?) they launched a $1 MM contest for data scientists to help them improve the accuracy of their black box.
Meanwhile they report their self-measured accuracy here: How Accurate Is The Zestimate? According to a statement on that page “Zestimate accuracy is computed by comparing the final sale price to the Zestimate on or before the sale date.” We will revisit that statement in a moment but for Chicago they are claiming that when looking at homes sold 44.3% of Zestimates end up within 5% of the final sales price and the median error is 5.9%. But we can also figure out from other data on that page that another 23.8% of the Zestimates end up being off by 5 – 10%, another 17.1% are off by 10 – 20%, and 14.8% are off by more than 20%. Of course, if you are a home buyer or seller you don’t know ahead of time how big the error is on the home of interest to you.
But back to the way Zillow measures their accuracy – looking at the sale price vs. the last Zestimate. That’s exactly the way I have independently measured their accuracy in the past. However, I now believe that they actually took the advice I gave them 1 1/2 years ago on how to easily make their accuracy LOOK better. In Those Home Value Estimators Really Suck I pointed out that a) there is no point in having an estimate above the current list price and b) there is no point in having an estimate much different than the sale price in the last 12 months. Not that I think they read my blog post and reacted accordingly. They may have but then again it doesn’t take a rocket scientist to figure out the obvious.
6 months after that post Zillow rolled out enhancements to their estimation algorithm that supposedly dramatically improved their accuracy. However, they do not publicly admit to making the obvious improvements I suggested – just much more sophisticated ones. Keep in mind that I just eyeballed it here, but when I spot check their accuracy the obvious, gross errors that I used to see are gone. The Zestimates for currently listed properties are usually not that far off from the list price nor are the Zestimates for recently sold properties. So it’s hard for me to believe that they left my low hanging fruit on the tree.
Perhaps the most obvious example is with recently listed homes. Spot checking them it looks like in many, many cases the algorithm often made significant adjustments coincident with the listing of the home. Here is an example of a Lake View home at 3726 N Marshfield. It was originally listed on September 6, 2016 for $700,000. Just prior to that the Zestimate was $620,000 but after the listing hit the market the Zestimate got bumped to $692,000 almost immediately. And look at the history of the Zestimate in the graph below. It had never undergone that big of an adjustment before.
There is only one problem with this method of improving the Zestimate accuracy. The only purpose is to avoid looking stupid – after the fact – and make the accuracy numbers LOOK better. It doesn’t really improve the underlying accuracy of the algorithm which ideally should be measured before any property specific, market based adjustments are applied. In other words the Zestimate is only useful if it can reasonably predict the selling price of a home before you actually put it on the market. And these adjustments don’t move that needle but they sure make the accuracy numbers look better than they were before.
#Zillow #Zestimate
Gary Lucido is the President of Lucid Realty, the Chicago area’s full service discount real estate brokerage. If you want to keep up to date on the Chicago real estate market, get an insider’s view of the seamy underbelly of the real estate industry, or you just think he’s the next Kurt Vonnegut you can Subscribe to Getting Real by Email using the form below. Please be sure to verify your email address when you receive the verification notice.