Are Zillow’s tools actually good for gauging home value?

Zillow bills itself as a leading real estate and rental marketplace, where users can go to learn a home’s real value, thereby arming themselves with the knowledge they need to be smarter buyers and sellers.

But recently, the online real estate giant is coming under fire, with multiple lawsuits claiming that their numbers may be so off, making them harmful to consumers.

In May, an Illinois real-estate lawyer filed a lawsuit against Zillow, stating that although Zillow cautions its visitors that a “Zestimate”—the company’s term for the numerical home value derived from a proprietary formula—isn’t the same as an appraisal, buyers don’t know the difference. Because of this, it has the same effect.

According to the Illinois woman’s suit, because the “Zestimate” is “promoted as a tool for potential buyers to use in assessing [the] market value of a given property,” it effectively meets the Illinois state definition of an “appraisal.”

The Zestimate for the woman’s home apparently undervalued her house so badly—the most recent Zestimate for her property values it at $64,000 less than she paid for it in 2009—that she says it has hurt her ability to sell the home.

This isn’t the first time the difference between Zillow’s Zestimate and the seller’s actual sale price has been significantly wide. As one media outlet noted, for a randomly-picked house outside of Washington, D.C., the gap between the Zestimate and the sale price is nearly $600,000.

Zillow Says The Issue Is With The Algorithm

Zillow has acknowledged that errors are possible. Nationwide, it has a median error rate of 5 percent: Zestimates are within 5 percent of the sale price 53.9 percent of the time, within 10 percent 75.6 percent of the time and within 20 percent 89.7 percent of the time, Zillow told The Washington Post.

While Zillow continues to defend the Zestimate, it recently launched a competition to encourage data scientists to improve its Zestimate algorithm. The Zillow Prize competition will take place in two rounds. From now until January 2018, teams can enter the public qualifying round by developing a model “to improve the Zestimate residual error.”

The teams with the prototypes that most narrow the gap will then advance to the second round, which runs from February 2018 until January 2019. While those 100 teams have to build an actual algorithm to do what the Zestimate does, the winner from that round gets $1 million.

Only time will tell whether Zillow’s accuracy improves but, in the meantime, it may be best not to use it as your primary source of data on market value.