1# RECORD 9c7b2eaba767769a138e2eee57dcf072ba93d6572# TITLE: From human business to machine learning—methods for automating real estate appraisals and their practical implications3# YEAR: 2022.0 VENUE: Zeitschrift für Immobilienökonomie DOI: 10.1365/s41056-022-00063-14# AUTHORS: Moritz Stang; Bastian Krämer; Cathrine Nagl; W. Schäfers5# PAGES: 28 CANDIDATE TABLE PAGES (1-based): [18]6# ABSTRACT: Until recently, in most countries, the use of Automated Valuation Models (AVMs) in the lending process was only allowed for support purposes, and not as the sole value-determining tool. However, this is currently changing, and regulators around the world are actively discussing the approval of AVMs. But the discussion is generally limited to AVMs that are based on already established methods such as an automation of the traditional sales comparison approach or linear regressions. Modern machine learning approaches are almost completely excluded from the debate. Accordingly, this study contributes to the discussion on why AVMs based on machine learning approaches should also be considered. For this purpose, an automation of the sales comparison method by using filters and similarity functions, two hedonic price functions, namely an OLS model and a GAM model, as well as a XGBoost machine learning approach, are applied to a dataset of 1.2 million residential properties across Germany. We find that the machine learning method XGBoost offers the overall best performance regarding the accuracy of estimations. Practical application shows that optimization of the established methods—OLS and GAM—is time-consuming and labor-intensive, and has significant disadvantages when being implemented on a national scale. In addition, our results show that different types of methods perform best in different regions and, thus, regulators should not only focus on one single method, but consider a multitude of them.789===== PAGE 1 (score 3, numeric-paren 0, table-words 0, var-words 3) =====1011ORIGINAL PAPER12https://doi.org/10.1365/s41056-022-00063-113Z Immobilienökonomie (2023) 9:81–10814From human business to machine learning—methods15for automating real estate appraisals and their practical16implications17Moritz Stang18· Bastian Krämer · Cathrine Nagl · Wolfgang Schäfers19Received: 4 December 2021 / Revised: 27 June 2022 / Accepted: 3 July 2022 / Published online: 19 July20202221© The Author(s) 202222Abstract Until recently, in most countries, the use of Automated Valuation Models23(AVMs) in the lending process was only allowed for support purposes, and not as24the sole value-determining tool. However, this is currently changing, and regulators25around the world are actively discussing the approval of AVMs. But the discussion26is generally limited to AVMs that are based on already established methods such27as an automation of the traditional sales comparison approach or linear regressions.28Modern machine learning approaches are almost completely excluded from the29debate. Accordingly, this study contributes to the discussion on why AVMs based30on machine learning approaches should also be considered. For this purpose, an31automation of the sales comparison method by using filters and similarity functions,32two hedonic price functions, namely an OLS model and a GAM model, as well33as a XGBoost machine learning approach, are applied to a dataset of 1.2 million34residential properties across Germany. We find that the machine learning method35XGBoost offers the overall best performance regarding the accuracy of estimations.36Practical application shows that optimization of the established methods—OLS and37GAM—is time-consuming and labor-intensive, and has significant disadvantages38when being implemented on a national scale. In addition, our results show that39different types of methods perform best in different regions and, thus, regulators40should not only focus on one single method, but consider a multitude of them.41Keywords Automated Valuation Models · Extreme Gradient Boosting · Housing42Market · Machine Learning · Sales Comparison Method43Moritz Stang () · Bastian Krämer · Cathrine Nagl · Wolfgang Schäfers44International Real Estate Business School, University of Regensburg, Regensburg, Germany45E-Mail: Moritz.Stang@wiwi.uni-regensburg.de46K474849===== PAGE 18 (score 9, numeric-paren 0, table-words 2, var-words 3) =====50519852Z Immobilienökonomie (2023) 9:81–10853Table 754Model prediction errors 2020 throughout Germany55Models56MAPE57MdAPE58PE(10)59PE(20)60R261EXF620.2130630.1624640.3267650.5872660.773567OLS680.1736690.1311700.3937710.6940720.865473GAM740.1646750.1202760.4273770.7276780.866479XGB800.1465810.1084820.4665830.7786840.899585the performance. The EXF is designed to replicate the practice of traditional real86estate appraisers in an automated manner and is therefore readily understandable.87However, the approach provides the poorest results. Comparing these results with the88performance of the OLS, often used as a baseline model, we can see a performance89improvement. Relatively speaking, the MAPE of the OLS is around 18% lower90and the MdAPE 19%. In addition, using an OLS results in 18% and 20% more91predictions deviating less than 10 and 20% from their actual market value.92Analyzing the results of the GAM, we again see a boost in performance compared93to the OLS. But this time the relative improvement is smaller. The MdAPE of the94OLS is around 9% higher. In addition, the percentage of predictions with a relative95deviation of less than 10 and 20% increased by 9% and 5% respectively. This might96be caused by the ability of the GAM to model more complex non-linearities within97the data, which is extremely difficult to manually reproduce within the OLS, and98practically impossible to implement for 327 districts. This is especially so, since99these manual adaptions have to be done in each of the four quarters.100Overall, the XGBoost yields the best model performance regarding all evaluation101metrics due to its ability to capture and process joint effects, non-linear relationships102and high-dimensional structures within the data with comparably low manual effort.103Comparing the results of the XGBoost with the EXF 43% and 33%, more observation104deviate less than 10 and 20% from their market values.105The chosen extending-window testing approach allows us to further analyze the106performance of all four algorithms over the four quarters of 2020. Confirming the107previous results, the solid line in Fig. 3 shows the trends already mentioned. Addi-108tionally, it is interesting how consistently the models perform over all four quarters.109Moreover, the XGBoost displays better performance the more training data it can110process. The exact numbers can be seen in Appendix III.111One research question of this study is to determine whether modern machine112learning methods are able to outperform traditional hedonic models and the EXF113approach. Analyzing our results at the national level for Germany we can clearly114confirm this. The XGBoost yields a significant performance improvement compared115to the EXF, OLS and GAM. This shows that in the future, regulators should also116discuss the approval of machine learning methods in the field of AVMs. The appli-117cation of machine learning approaches can lead to a reduction in the economic loss118caused by the AVM. Machine learning algorithms are able to better assess possible119risks within the lending process and can thus fulfill the actual purpose of a real120estate valuation in a much more target-oriented manner.121K122123124===== PAGE 19 (score 2, numeric-paren 0, table-words 0, var-words 2) =====125126Z Immobilienökonomie (2023) 9:81–10812799128Fig. 3 Quaterly model error129performance based on MAPE1305.2 Results at the administrative district level131After comparing the models at the national level, we want to examine the model132performance in more detail. Therefore, we focus on the level of the 327 adminis-133trative districts. In Fig. 4, the performance based on the MAPE for the different134methods is shown cartographically. The maps confirm the abovementioned trends.135The EXF again yields the overall poorest performance and again, it can be seen that136the more complex the approach, the better the results. In addition, all four models137are unsatisfactory with respect to estimating the market value in the same admin-138istrative districts. This can also be confirmed by the correlation matrices shown in139Appendix III. Especially in the eastern part of Germany, the MAPE tends to be140higher. This result might be caused by the lower data availability in these regions.141To obtain a better understanding of the model performance at the administrative142district level, we focus on the box plots of the MAPE in Fig. 5. Those confirm143the trend displayed in Fig. 4. The EXF again yields the overall poorest results. It144delivers the largest interquartile range, the longest whiskers and contains the most145outliers. The XGBoost has the lowest median MAPE of all four models, whereas it146has only two extreme outliers. In contrast, the GAM and especially the OLS have147a smaller range of outliers. These results indicate that the XGBoost does not always148Fig. 4149Error comparison at administrative district level150K151