Collateral Valuation Reimagined: The Computer Vision Advantage

This month the AVMNews sat down with Nathan Brannen for an Exclusive Interview. For those of you who have not had the pleasure of meeting Nathan, he is the Chief Product Officer at Restb.ai, the leading provider of AI-powered solutions to the real estate industry, and has specialized in applying computer vision and AI to collateral valuation.

With over a decade of experience in developing and launching AI-based products across the energy and real estate sectors, Nathan has been instrumental in expanding Restb.ai's market presence, facilitating partnerships with many of the leading firms in the appraisal and valuation space, as well as becoming the standard AI solution for the industry's MLS technology platform providers, reaching over 700,000 real estate agents and 10,000 appraisers and valuation professionals.

1. Revolutionizing AVMs with Computer Vision

AVMNews: Nathan, as someone who is rooted in the Automated Valuation Model (AVM) Industry, you have witnessed firsthand the evolution of the technology and data availability. Could you share insights into the new techniques and technologies you have recently implemented, observed, or plan to implement?

Nathan: As an AI data-driven company that partners with many AVM companies, rather than marketing our own AVM to others, we benefit from seeing, collaborating, and exploring many exciting and new ideas that are being explored by the leading innovators in the space.

AI's Role in Condition and Quality Scoring

Most of our unique insights pertain to leveraging computer vision insights at scale to improve valuation accuracy. To do this, I think it’s most appropriate to bucket the areas we’ve focused into two distinct areas: Condition/Quality Scores and Property Characteristics.

Let’s start with the most impactful area we’ve seen, condition and quality scores generated via computer vision. If you look at the main price drivers of a home’s value, the first three typically are a property’s location, livable area, and lot size, which are widely and accurately available. On the other hand, the next two, a home’s condition and quality, are often blind spots for AVMs. Due to their importance, many AVM companies look for proxies of a home’s condition and quality, such as by parsing listing remarks for clues, estimating based on age or monitoring building permits. While these approaches may prove better than random, they are susceptible to bias and gaps (i.e. agents may “generously” market their property, older homes can be renovated, not all renovations require permits, etc.) that can negatively impact an AVM’s overall performance.

Granular Scoring Beyond Standardized Scales

When we developed our photo-based condition and quality models 5 years ago, our goal was to provide a standardized score that could be used to reliably compare properties against one another. We initially modeled our scores after the GSE’s 6-point UAD scale but quickly realized that a more granular approach was needed. For example, looking across millions of appraisals, we see approximately 85% of properties are the two middle condition ratings, a C3 or a C4, and 96% of properties are scored with a quality of a Q3 or a Q4. This makes sense as it’s hard enough to convince thousands of appraisers to treat properties objectively and consistently across various, diverse markets around the country. On the other hand, we intuitively realize that condition and quality are both spectrums, and there may be significant value adjustments between properties that may fall into these broad industry-standard buckets.

Fortunately, AI can provide unparalleled consistency and granularity at scale. We built models to analyze a property to one decimal place (i.e. a C3.4), increasing the number of scores from 6 to more than 50. Additionally, AI allows us to generate scores for not only the overall property, but also the four main areas of a home: the exterior, interior, kitchen and bathroom. This breakdown allows AVM models to understand the nuance of a home with a consistent condition throughout versus a property with a kitchen renovated last year but bathrooms that have been untouched for decades. Layering these scores into an AVM significantly improves the overall MAE and decreases PPE10, as properties that may previously have been assumed to have an average condition or quality can have their values correctly adjusted in the appropriate direction.

2. Overcoming Challenges in AI Integration

Moving on to using AI to analyze property characteristics, we also look at the features, materials, amenities, etc., that can be detected within a property’s photos. Many of these items may already be present in traditional sources, such as listing or public record datasets. However, although this data may technically exist, there are three main opportunities where supplementing the existing data with what computer vision detects can improve AVMs.

Addressing Data Inconsistencies and Gaps 

The first challenge is the well-known, but accepted fact that neither listing or public records data are perfect. These datasets are frequently incomplete for various fields in certain regions. In other cases, they may contain incorrect or, at best, misleading data. We recently did an analysis across a variety of fields, comparing listing values to the same insights generated by AI from photos and found many cases where AI could fill these gaps. In one example, positive features like a property having an indoor bar or an outdoor kitchen were surprisingly underrepresented in MLS listings by more than 60% and 80%, respectively. In another case, we saw that AI could determine whether a basement was finished, partially finished, or unfinished in 88% of the cases where an agent had recorded a property having a basement but not specified its status.

Standardizing Diverse Data Sources

The second reason is that the fields in these datasets are not standardized. Listing data is commonly aggregated across hundreds of MLSs, while public records are collected from thousands of counties. Despite the best efforts by organizations like RESO to standardize things, the number and type of data fields and enumerations vary across these markets. It takes an immense amount of effort to clean this data, which is required prior to utilizing it effectively in an AVM. For example, different MLSs may record the presence of “RV Parking” in various fields such as, “Exterior Features” or “Parking Features” and with limitless different variations, such as RV Parking, RVParking, Existing RV Parking, RV/Boat Space, etc.

Managing Complex Data Sources

The last opportunity results from the definition of various property characteristics within these datasets that are not standardized. When a traditional dataset mentions that a property has a patio, is it referring to a 6’ x 6’ concrete landing in the backyard or a 350 sq ft sprawling brick patio, perfect for hosting a large event? Moving to more subjective elements, like a property’s architectural style, it becomes even more difficult. When we analyzed all of the properties listed in the MLS as A-Frame style homes, we estimated that only 4% fit the true definition of an A-Frame.

This lack of consistency and completeness within many of the core datasets makes it incredibly difficult for AVMs to reliably determine the relevant price impact of many data points. In fact, I believe that many data points that had previously been written off as non-factors in a property’s value may actually be relevant if they could be analyzed as part of a complete dataset. Fortunately, AI can effortlessly generate these insights at scale and provide them as plug-and-play inputs for an AVM, with no cleaning necessary.

AVMNews: Have there been any new techniques that you explored but eventually discarded due to being less effective than anticipated, overly costly, or presenting other disadvantages?

Nathan: In my prior response, I mentioned how AI could generate data without cleaning it. This needs clarification. On our end, we spend a phenomenal amount of time and effort trying to understand what these AI-generated data points look like at scale. Whenever we run into a scenario where a data point that we would intuitively expect to improve an AVM has a negative or negligible impact, we have to dive into the examples to determine the cause of the variance. To make most of our visual insights reliable for an AVM, there is an immense amount of analysis and cleaning of corner cases to ensure when the data is delivered to a client, it just works.

As an example, when we first started collaborating with clients to layer this data into their AVMs, we tested a variety of markets to evaluate the impact of our condition and quality scores. In the majority of markets, the results were encouraging, but in two particular markets, the results just weren’t there.

Lessons from Market-Specific Challenges

In the first case, we were analyzing San Diego and the impact of the condition score was negligible. Upon a closer look, we discovered that many new construction homes were being sold prior to the construction being finished. As such, the images included in the listings were often of an unfinished structure, which our model was scoring as a C6.0 or an “unlivable” condition. However, the price associated with this property was consistent with other properties that were scored as C1.0, or a “new” condition. This massive disconnect made it difficult for the AVM to converge and consistently understand how the condition score of a property should be utilized to predict a property’s value.

In the second example, we looked at Vancouver and noticed problems with the impact of a property’s quality. Vancouver is unique compared to most other cities in that approximately 50% of its housing stock is condos. When analyzing the listing photos of these condos, it was customary to include photos of the common areas, such as an expansive lobby with a high ceiling or a rooftop gym with beautiful views of the skyline. These photos inevitably scored well in our quality model, improving the overall property score compared to the “actual” quality that would have been determined by the comparatively less impressive photos of the actual condo unit.

While the inclusion of atypical or problematic images in certain listings has been the cause of many of the “disappointing” results we’ve encountered, once the root cause has been identified, it is typically straightforward to train the AI to adapt accordingly. In fact, at this point, I think the majority of future improvement that can come from leveraging AI-generated visual insights in AVMs will come from better understanding and handling of these corner cases rather than improving the raw ability of how well AI can identify a particular feature or concept in a photo.

3. The Future of AVMs: Trends and Innovations

AVMNews: Amid the industry's excitement about Artificial Intelligence integration and skepticism towards Machine Learning in AVM products, could you elaborate on the roles that machine learning and artificial intelligence play in enhancing AVM technology?

Nathan: Again, AI has the ability to generate data in a granular and consistent way that is unmatched by current processes. It is simply not feasible to ask agents to spend more time entering in property data to a listing nor to provide new additional data points like condition or quality in a standardized, AVM-friendly format. Computer vision solves this challenge, generating new, expansive datasets in real time. If a new concept is of interest, you can back-propagate new data points across historical properties in a way that would never have been possible in the past.

For example, concepts like kitchen layouts (e.g. galley, L-shaped, etc.), damages, dated components (cabinets with visible hinges, old stylistic elements, etc.), and more can now be examined across every property. Even the weather (i.e. sunny vs. overcast) when listing photos were taken can be tracked and analyzed. While this may seem like creating data for data’s sake, the reality is that many of these insights have a statistically relevant impact. We’ve proved that listings that include overcast photos have a median DOM of more than 10% longer than comparative listings, with some markets seeing significantly larger impacts.

Another benefit of this standardized, structured data generated at scale is that it can deliver market level insights that can be provided as supplemental context alongside an AVM value. Whereas before, it was common to include a property’s price per square foot (ppsf), AI can also provide the ppsf of a property’s unique market based on recent sales across different condition and quality bands. For example, the ppsf of a C3 (i.e. a property featuring normal wear and tear) may be $350 in a certain neighborhood while a C4 (i.e. a property in need of minor updates) may be $250. In a bordering neighborhood, perhaps the ppsf of C3s are $450. These values can be generated for any market to provide up to date insights on how any property fits within its unique market, which is invaluable context to understand a property’s valuation more completely.

The reality is that AI is enabling a rapid increase in the amount of data that exists. It will take time for this data to be analyzed, cleaned, and responsibly utilized at scale. Furthermore, since real estate is hyper local, it will also take time to understand how this data varies and impacts markets differently. With so many new data points to explore, I believe we’re only scratching the surface of how they can be incorporated into AVMs to improve valuations.

AVMNews: Looking ahead, where do you foresee the next wave of improvements arising in the realm of AVMs?

Nathan: I’ve recently been fascinated by two concepts that I believe open up a new dimension to how we view AVMs. The first is the idea of quantifying supply/demand dynamics and how they impact a property’s price at any point in time; the second is a better understanding of a property’s complexity within its unique market.

Exploring Supply and Demand Dynamics

Concerning supply/demand, you’ll sometimes hear agents say that a house is worth what someone is willing to pay for it, while an AVM attempts to predict the expected value a home is worth. My belief is that homes are worth different values to different people, and we’re not so far away from an AVM being able to take that into consideration.

Starting with the demand side of the equation, we are entering a world where we not only know what listing users are looking at but also the specific images or elements of a property that are most interesting to them. It is now possible to track the photos a potential home buyer is looking at to determine the particular features they care about the most. The amount of time they spend on different photos provides a new depth of understanding of their interests. Perhaps more importantly, as more portals introduce natural language search queries, more data is generated on what buyers are looking for, regardless of whether it exists in the market or not. All of this data can be aggregated up and compared across time to see if there are spikes for certain types of properties, which could lead to seemingly inflated prices.

It is a similar story on the supply side of things. Imagine you know that historically, there are three modern homes sold each month in a particular neighborhood of a city, but at this particular point in time, there are currently no modern homes listed. In theory, an off-market modern home in that area would be worth more at that point in time than it would be otherwise.

Understanding Property Complexity and Uniqueness

The other aspect I mentioned was property complexity. While products that analyze how unique a property is compared to recently sold properties already exist, they have historically only focused on high level property characteristics like size, location, beds/baths, etc. Utilizing visual insights in these models opens up a new world of possibilities to better identify when a property may be complex. For example, knowing a property is much nicer or in a much poorer condition than the other similarly sized homes in its neighborhood will result in a better understanding of how its uniqueness may impact its price. In a more humorous but no less relevant example, knowing a property has lime green walls, a shag carpet, a sunken living room, or any other feature that may appeal to a smaller subset of the population will help reveal how much trust can be given to an AVM’s predicted value.

4. Advancements in AI for AVMs

In both cases, the vast amount of new data that AI will generate can be leveraged by AVMs to not only improve their valuations, but also provide critical context on how to interpret the provided value and value range.

AVMNews: Engaging with clients regularly to provide a variety of AVM solutions, you have unique insights. What aspects of AVM development and enhancement do you believe are undervalued by those outside the industry with limited access to insider knowledge?

Nathan: While there will always be complaints that AVMs aren’t perfect, I think there isn’t as much appreciation for how good they’ve become in the US, particularly compared to other markets. We work with clients in more than 35 different countries, and AVMs either don’t exist or are not widely utilized in almost all of them. This is mainly due to issues with companies being unable to access sufficient data or the existing data being so unreliable as to be actively unhelpful (i.e. recorded living areas frequently being in +/-20%). With AVMs being so ubiquitous in the US, I think people would have a hard time imagining a world without them.

AVMNews: Is there any specific aspect of your AVM or Validation product, your approach, your Cascade Modeling/ or product offering that you would like to emphasize? What sets it apart or contributes to its inherent strengths?

Introducing Confidence Scores for Enhanced Accuracy

Nathan: A recent development we’re excited about concerning AVMs is our new confidence scores for our condition and quality scoring models. As we looked more into where our scores were creating problems for our clients at scale, it became readily apparent that the larger errors were almost entirely related to listings including an atypical set of photos. This could include listings that only had one photo, listings where the photos were low quality, listings that included photos that had been heavily digitally altered, etc.

With our recently released confidence scores, we now provide a score from 0-100 to specify the score’s reliability. Additionally, we provide a fixed set of explanations that detail why the confidence score was low or the property was atypical. The response was explicitly designed to be able to be programmatically passed as an input to AVMs so they can learn when and how to trust the condition/quality scores we provide in these unique situations.

Many clients ask us if they should set a cutoff to determine when to use or not use our scores, but I think this is a suboptimal approach. If our AI provides a score with a low confidence because there was only one exterior photo included with a listing, it doesn’t mean that the score the AI provided is wrong, it simply means that not knowing whether the interior had just been fully renovated or if it was in disrepair could shift the provided score up or down. The low confidence reflects the higher possible variance that could exist related to a property’s condition or quality.

AVMNews: Would you like to share any thoughts about the impending AVM quality control rule?

Closing Thoughts and Industry Reflections

Nathan: I believe computer vision will be an essential component for compliance with the new AVM quality control rules. For example, in addition to providing our photo-based condition and quality scores as inputs to an AVM, it is also possible to give the positive or negative components (i.e. premium materials, dated components, etc.) that provide explainability to the AI-generated score. In addition to a numerical value, our confidence score provides context (i.e. too few images, low image quality, excessive clutter, etc.) for why a property’s score should be taken with a grain of salt. Perhaps more importantly than ever, our models can ensure that the listing photos included in a property have not been digitally altered.

When it comes to being impartial and fair, there are understandable concerns that irresponsible use of AI may perpetuate historical biases. However, it is equally important to acknowledge how certain types of AI, such as computer vision, can be used to counteract or detect possible bias. When we train our models on a concept like condition or quality, we analyze the property without any knowledge of the property’s sale price or owner. We’ve also built out models that can detect people and any indicators of ethnicity, political or religious affiliation. Using these models, we can audit our training datasets to ensure there is no bias towards a particular class. We also make these available for clients to detect and blur these possible bias items before they pass images to our condition and quality models to ensure every property is viewed objectively.

5. Closing Thoughts

AVMNews: Nathan, thank you for engaging with the AVMNews Community through this interview. Your insights on incorporating AVMs intelligently and exploring new avenues such as using computer vision for enhanced efficiency and elaborating on topics that resonate with our readers and align with the core mission of the AVMNews. We are thrilled to have you as an industry expert and professional in the field of AVM technology. We hope discussing these nuanced and exciting aspects of AVMs can truly foster meaningful conversations within the industry.

Connect with Nathan: Nathan Brannen on LinkedIn.

This article was originally published as part of AVMetrics newsletter series: AVM News, Volume 23, Issue 12. December 2024.