For nearly a decade Lee Kennedy commandeered nuclear submarines for the U.S. Navy and when the Cold War ended, he sought a different path to apply his education in nuclear engineering. Kennedy landed a job at American Savings Bank, later acquired by Washington Mutual, and he helmed the company’s fledging Alternative Valuation Business Unit.

By the late 1990s, Kennedy said, data storage and computing power was growing cheaper. Washington Mutual felt increasingly comfortable using data points about a single-family home – when the property was built, its location, available purchase price history – as fuel powering a valuation model to extend someone a home equity line of credit.

Fast forward to today, and alternative valuation models – redubbed automated valuation models or AVMs – are deployed to extend credit, appraise a home, and – in the case of iBuyers – even guide when to purchase a home for cash that might be profitable to resell.

When Zillow waved the white flag on its iBuying operation earlier this month, CEO Rich Barton couched the company’s price forecasting model as something of a Frankenstein’s monster, an audacious curiosity that proved too dangerous. “Our observed error rate has been far more volatile than we ever expected possible,” Barton said.

To some housing executives who routinely use AVMs, Barton’s explanation rang false. “This is not a problem with valuation models,” said Matt Woods, CEO of real estate consulting service Sold.com. “This is a Zillow problem.”

But to Kennedy, who for the last 16 years has run AVMetrics in Simi Valley, California, an evaluator of AVMs for client companies, what happened at Zillow is not surprising.  

What’s different about Zillow? “Most of my clients are large lending institutions and shy away from publicly admitting a mistake,” Kennedy said.

With Zillow out, there are two publicly traded companies, Opendoor and Offerpad, whose predominant business model is iBuying.

Both are scaling up rapidly. Opendoor posted a 91% revenue increase to $2.3 billion in its quarter three earnings report released Wednesday. Offerpad revenue grew 30% to $540 million in the third quarter.

But neither has found a profitable path even amid a booming housing market. Opendoor lost $56.8 million in the quarter. Offerpad posted a $15.3 million loss, though it was profitable in the second quarter.

Kennedy is skeptical that a pricing model can ever work for Opendoor, Offerpad or future iBuyers. Such a model, the engineer pointed out, must assess current value while baking in near-term price fluctuations.

“Inflection points in the market are hard to predict,” Kennedy said.

How a home pricing model works

Clifford Rossi, now a finance professor at the University of Maryland, recalls working at risk management in Fannie Mae and Freddie Mac when, after connecting to Netscape on a dial-up modem, employees could access a proprietary data storage system of county property records.

That storage of those records begat one of the first home pricing algorithms, initially used in the late 1990s as guideposts for bundling mortgages. By 2004, Rossi said, Fannie Mae and Freddie Mac greatly expanded the application of AVMs via the “property inspection waiver” by which mortgage lenders could “streamline operations” by using an AVM appraisal output, instead of a human appraisal, to issue a loan.

When the housing bubble burst, pricing model use was rolled back. “But since about 2012,” Rossi said. “There has been a little bit more relaxation in how the AVMs get applied.”

This permissiveness coincided with the growth of AVM vendors such as CoreLogic and HouseCanary. It also dovetailed with the advent of iBuyers Opendoor (founded in 2014), Offerpad (2016), plus power buyers, companies like Orchard and Ribbon that facilitate a consumer switching homes by making a cash offer on properties.

That Zillow could both fail at iBuying and blame their pricing model strikes these proptech executives as a Zillow problem and not a rebuke of pricing models.  

“AVMs are built to help rational decision making, but Zillow was manually overriding its algorithms,” said Court Cunningham, CEO of Orchard, a New York-based company that claimed a $1 billion valuation last month.

Faulting the humans for defying the machines was echoed by other close Zillow observers, plus a Business Insider article that reported on the “over exuberance of human managers at the company.”

(Zillow declined to comment on the question of whether it overrode the algorithm. The company mostly responded to questions by referring back to its public statements. A spokesperson also noted that the still extant “Zestimate” is a “starting point” for market value, too challenging to forecast home prices three-to-six months out.)

Orchard’s AVM also uses machine learning, a form of artificial intelligence focused on pattern recognition. Machine learning increases the number of property comparisons – the “comps” that form the backbone of a human appraisal – from three-to-four to dozens.  

Home pricing AVMs have also evolved to input what might seem like intangible factors, said Matthew O’Hara, head of portfolio management and research at Unison Investment Management.

Unison contributes to the investment of a person’s home, and in exchange gets access to the person’s equity. In order to determine what homes to invest in, Unison accesses AVMs with features like noise indexes, proximity to transportation, and school and crime data.

But AVMs are less equipped to forecast future home prices.

“The AVM is a product based on today’s value,” said Winfield Xu, a senior data scientist manager at Unison. “A home value forecast is an economic model that usually only goes down to the zip code level.”

“A home pricing forecast looks at underlying house pricing fundamentals, data from Moody’s, local employment trends,” Rossi said. “It’s more like betting on a stock since it’s hard to capture turns in the economy.”

That makes it incumbent on iBuyers to resell before the economy turns.

“Ibuyers are in the business of buying low and selling high over a short period of time,” said Greg Buchak of Stanford University, who has studied the industry. “They are not out there trying to predict the economy. They do not want to get stuck holding a bunch of houses.”

Opendoor’s home pricing model

In a November 2020 Securities and Exchange Commission filing before the company went public, Opendoor laid out some of the features of its AVM.

The company collects data from Multiple Listings Services; MLS data incorporates facts like square footage plus more open-ended characteristics such as “hardwood floors throughout.”

Opendoor also inputs “proprietary data assets.” This includes non-public data about the homes’ underwriting history as well as “seller input,” which is the online questionnaire potential home sellers fill out. The company’s filing also mentions, “Geospatial data assets, such as power line proximity and road noise level.”

These are bullet points, or one-off sentences in Opendoor’s pitch to prospective investors. They do not explain why a seller would be motivated to be completely forthright in a questionnaire, or how variables are weighted, or how Opendoor’s AVM is different from others, including Offerpad’s, which months later released a similar presentation.

The public filings examined also say little about price forecasting, the element of Zillow’s iBuying model that company CEO Barton specifically said is too volatile.

Opendoor has declined to comment on questions about its AVM, noting it doesn’t disclose specifics of its pricing algorithm because they are proprietary.

Still, Opendoor supporters view the company as the iBuyer that knows what it’s doing. The day after Zillow announced its wind down, Opendoor co-founder Keith Rabois, who is no longer part of company leadership, tweeted: “If you read our SPAC filings, we explained how our algorithms actually work vs Zillow’s which are horrible.”

Flaws in pricing models

Though Matthew O’Hara at Unison uses AVMs, the company’s decision on what homes to invest in are driven by an independent contractor, third-party appraiser.

“It is partly a trust issue,” O’Hara said. “The homeowners that we are entering into a contract with understand better the methodology.”

In addition to demystifying the home valuing process, the human appraiser fact-checks what the automated pricing model inputted.

“In public records, a home might say it has five bedrooms when really it has three,” Xu of Unison said. “AVMs are 100% trusting of public data and sometimes that public data is wrong.”

AVMs can also fall short in evaluating a home’s condition or the surrounding neighborhood. “There could be shag carpeting that has been around since the 1970s,” Rossi, the Maryland professor, said. “There could be a run-down, abandoned home on the block.”

These issues can generally be addressed after in-person inspection, which Opendoor, Offerpad and power buyers conduct.

But they throw into question the accuracy of computer-generated comps: HouseCanary CEO Jeremy Sicklick has claimed the company uses up to 500 comparable homes in a valuation. But which of those comps has secretly ugly carpet? “There are still Achilles heels to price modeling,” Rossi said.

Shaival Shah, CEO of Ribbon, said his company is aware of such shortcomings. “Every home is a unique snowflake with a really interesting complexity,” Shah said.  

Ribbon uses a three-pronged approach – a real estate agent visiting the property it may buy, a “desktop” appraisal from a Ribbon employee, and a home value that’s the average from numerous AVM vendors– in order to find a valuation.

Power buyers like Ribbon and Orchard, however, are buying a home for cash that their client has already agreed to move into. Ibuyers, meanwhile, typically buy a home without the future homeowner in mind.

“It is very different,” Cunningham of Orchard said, “From a risk standpoint.”

The iBuyer dilemma

Observers like Shah contend iBuyers increasingly seek corporate investors to buy their homes in bulk. Zillow is taking this approach with its remaining inventory. New York-based Pretium Partners will buy 2,000 homes from Zillow and rent them out, the Wall Street Journal reported earlier this month.

But most available evidence points to iBuyers selling homes the same way your neighbor does.  

“IBuyers use traditional selling channels, relying on Multiple Listings Services to dispose of their inventory,” asserted a December 2020 study by Buchak of Stanford, Gregor Matvos of Northwestern University, and Tomar Piskorski of Columbia University, one of the few academic studies on iBuying.

“Ibuyers are no better at selling houses than other households,” added the study (which is titled, “Why is intermediating houses so difficult? Evidence from IBuyers”). “Formally, their current matching technology is almost identical to other sellers. This is intuitive since they have to sell their houses through a listing process.”

For Zillow, that included pricing the inventory. An examination of Zillow homes languishing on the Phoenix area MLS indicates that the company was not adjusting to moderate, short-term trends like people buying fewer homes in the back-to-school period. By comparison, Opendoor and Offerpad modestly recalibrated purchase, and subsequently, resale price.

“Zillow’s model was not bad, but it lacked predictability,” said Kennedy of AVMetrics. “That’s where their chief economist comes in.”

But what if there is something more than the change of season to forecast – A rapid surge or diminishment in covid cases, an interest rates spike, or millennials deciding home ownership is overrated?

IBuyers, the study from Buchak, Matvos and Piskorski found, suffer a dilemma.

Their raison d’etre is making a traditionally illiquid asset – the house – liquid. But empirically, the homes that they can quickly buy and resell for a profit are typically “cookie-cutter homes” in suburban, Sunbelt enclaves. Homes without character, or, less pejoratively, homes without quirks that can mess up an AVM just enough to cause an iBuyer to pause. (Or, in the case of Zillow, mistakenly move full steam ahead.)

In other words, homes that are easy to buy and resell to begin with.  

“Intermediation is only profitable in the most liquid and easy to value houses,” the study reads. “Therefore, iBuyers technology allows them to supply liquidity, but only in pockets where it is least valuable.”

Opendoor and Offerpad are working to address these issues.

Carrie Wheeler, chief financial officer of Opendoor, noted on Wednesday’s earnings call the company has expanded into over 40 markets, increasing its “buy box” – that is the range of homes the company feels comfortable purchasing. Opendoor has expanded its buy box without suffering the exacerbation in net losses that befell Zillow.

Offerpad, meanwhile, has added value to its inventory through in-house renovators who refurbish homes, aiming to make the house they market a more valuable asset than the one they bought.

Going forward, both fast-growing companies must make difficult decisions of what markets they, and their AVM, feel comfortably expanding into.

“The iBuyer will always have a smaller buy box,” said Shah of Ribbon. “When they buy a home, they are immediately on the clock.”