1# RECORD a483774defcbadf7676aaa742f594af28562b13a2# TITLE: Predictability of Belgian residential real estate rents using tree-based ML models and IML techniques3# YEAR: 2023.0 VENUE: International Journal of Housing Markets and Analysis DOI: 10.1108/ijhma-11-2022-01724# AUTHORS: Ian Lenaers; K. Boudt; L. De Moor5# PAGES: 27 CANDIDATE TABLE PAGES (1-based): [10]6# ABSTRACT: 7Purpose8The purpose is twofold. First, this study aims to establish that black box tree-based machine learning (ML) models have better predictive performance than a standard linear regression (LR) hedonic model for rent prediction. Second, it shows the added value of analyzing tree-based ML models with interpretable machine learning (IML) techniques.91011Design/methodology/approach12Data on Belgian residential rental properties were collected. Tree-based ML models, random forest regression and eXtreme gradient boosting regression were applied to derive rent prediction models to compare predictive performance with a LR model. Interpretations of the tree-based models regarding important factors in predicting rent were made using SHapley Additive exPlanations (SHAP) feature importance (FI) plots and SHAP summary plots.131415Findings16Results indicate that tree-based models perform better than a LR model for Belgian residential rent prediction. The SHAP FI plots agree that asking price, cadastral income, surface livable, number of bedrooms, number of bathrooms and variables measuring the proximity to points of interest are dominant predictors. The direction of relationships between rent and its factors is determined with SHAP summary plots. In addition to linear relationships, it emerges that nonlinear relationships exist.171819Originality/value20Rent prediction using ML is relatively less studied than house price prediction. In addition, studying prediction models using IML techniques is relatively new in real estate economics. Moreover, to the best of the authors’ knowledge, this study is the first to derive insights of driving determinants of predicted rents from SHAP FI and SHAP summary plots.21222324===== PAGE 1 (score 0, numeric-paren 0, table-words 0, var-words 0) =====2526 27Vrije Universiteit Brussel28Predictability of Belgian residential real estate rents using tree-based ML models and IML29techniques30Lenaers, Ian; Boudt, Kris; De Moor, Lieven31Published in:32International Journal of Housing Markets and Analysis33DOI:3410.1108/IJHMA-11-2022-017235Publication date:36202337License:38CC BY-NC39Document Version:40Accepted author manuscript41Link to publication42Citation for published version (APA):43Lenaers, I., Boudt, K., & De Moor, L. (2023). Predictability of Belgian residential real estate rents using tree-44based ML models and IML techniques. International Journal of Housing Markets and Analysis.45https://doi.org/10.1108/IJHMA-11-2022-017246Copyright47No part of this publication may be reproduced or transmitted in any form, without the prior written permission of the author(s) or other rights48holders to whom publication rights have been transferred, unless permitted by a license attached to the publication (a Creative Commons49license or other), or unless exceptions to copyright law apply.50Take down policy51If you believe that this document infringes your copyright or other rights, please contact openaccess@vub.be, with details of the nature of the52infringement. We will investigate the claim and if justified, we will take the appropriate steps.53Download date: 15. Dec. 2023545556===== PAGE 10 (score 38, numeric-paren 0, table-words 2, var-words 22) =====5758nternational Journal of Housing Markets and Analysis59760Table I: Summary statistics for the target variable and features61Feature name62Data type63Nr. of 64observations65Mean66Standard 67deviation68Skewness69Correlation 70with target71Monthly rent (indexed to 2019) in EUR72Continuous731893574887.38475430.668764.27077/78Asking price (indexed to 2019) in EUR79Continuous801883181295106.97182155702.016833.409840.81285Cadastral income (in base year 1975) in EUR86Continuous87627088941.81589745.955905.579910.61492Energy Prestation Certificate (EPC; A = 0, B = 931, C = 2, D = 3, E = 4, F = 5)94Ordinal95830296/97/98/99/100Furnished (0 = no, 1 = yes)101Binary10258961030.1201040.3251052.341106-0.019107Garage (0 = no, 1 = yes)108Binary109160671100.2641110.4411121.0661130.079114Garden (0 = no, 1 = yes)115Binary116140521170.3201180.4671190.7711200.098121Land area in m2122Continuous12311054124478.7711251088.8581267.3191270.087128Min. distance to doctor (in degrees)129Continuous130189351310.0231320.0291331.843134-0.126135Min. distance to hospital (in degrees)136Continuous137189351380.1601390.1131400.965141-0.058142Min. distance to mall (in degrees)143Continuous144189351450.1961460.1551470.795148-0.172149Min. distance to park (in degrees)150Continuous151189351520.1331530.0801540.748155-0.040156Min. distance to restaurant (in degrees)157Continuous158189351590.0081600.0101612.238162-0.050163Min. distance to school (in degrees)164Continuous165189351660.0231670.0251681.916169-0.101170Min. distance to station (in degrees)171Continuous172189351730.0241740.0271752.315176-0.057177Min. distance to supermarket (in degrees)178Continuous179189351800.0171810.0201821.718183-0.057184Min. distance to university (in degrees)185Continuous186189351870.2561880.153189-0.126190-0.0127191Nr. of bathrooms192Integer193169241941.1491950.4511963.9051970.506198Nr. of bedrooms199Integer200158172012.3492021.0122030.6892040.434205Nr. of doctors in 5.5km range206Integer2071893520818.66920926.9502101.4612110.226212Nr. of hospitals in 5.5km range213Integer214189352150.3882161.0622173.6002180.094219Nr. of malls in 5.5km range220Integer221189352220.2932230.5522241.7362250.191226Nr. of parks in 5.5km range227Integer228189352290.2312300.5222312.2132320.020233Nr. of restaurants in 5.5km range234Integer23518935236136.421237208.0782382.2912390.253240Nr. of schools in 5.5km range241Integer2421893524311.21324413.7062451.2962460.244247Nr. of stations in 5.5km range248Integer249189352505.2342519.8992524.7202530.184254Nr. of supermarkets in 5.5km range255Integer2561893525741.40725868.0172592.2012600.253261Nr. of universities in 5.5km range262Integer263189352640.3682651.3582667.9672670.089268Number of toilets269Integer270112962711.2462720.7452731.0742740.395275Property type (0 = house, 1 = apartment)276Binary277189352780.6942790.461280-0.844281-0.202282Surface livable in m2283Continuous28417263285122.99928670.0872873.8842880.616289Surface of garden in m2290Continuous291682729228.857293163.53229415.0042950.093296Terrace (0 = no, 1 = yes)297Binary298146982990.7753000.418301-1.317302-0.012303Zip code304Nominal30518935306/307/308/309/310Source: own elaboration of the provided data311ML models312Tree-based (ensemble) models seem to have a slight edge in the literature considered, therefore RFR and 313XGBR were selected as ML models for this study. Both are compared on predictive performance to a LR 314model, estimated with OLS, which serves as a baseline model.315Tree-based regression techniques are based on individual Regression Trees (RT), which can identify 316complex patterns such as non-linear relationships and interactions (Antipov & Pokryshevskaya, 2012). 317Page 8 of 25318International Journal of Housing Markets and Analysis319132023213322432353246325732683279328103291133012331133321433315334163351733618337193382033921340223412334224343253442634527346283472934830349313503235133352343533535436355373563835739358403594136042361433624436345364463654736648367493685036951370523715337254373553745637557376583775937860379380381===== PAGE 11 (score 2, numeric-paren 0, table-words 0, var-words 2) =====382383nternational Journal of Housing Markets and Analysis3848385However, individual RTs are prone to misspecification, overfitting and there is as such little generalization 386(Provost & Fawcett, 2013). Therefore, tree-based ensemble techniques are used, which address these 387drawbacks while still being able to capture the complex relationships. These ensemble models create several 388sub models based on RTs. With the predictions of the sub models, the ensemble models combine the different 389predictions, of rent in this study, to obtain a final prediction (Hastie et al., 2009). RFR and XGBR both 390belong to different groups of ensemble models, which are called bagging and boosting, respectively.391RFR is a bagging ensemble method that combines predictions from several individual RTs into a final 392prediction. In parallel RTs are constructed. Random variation is introduced by selecting a subset of the data 393along one side for constructing the RT and along the other side by randomly selecting features for 394constructing branches of the RT (Hastie et al., 2009). From the predictions of different RTs, the average is 395taken to provide a final prediction. For more technical information on RFR, see Hastie et al. (2009).396Unlike RFR, which builds the forest of RTs in parallel, XGBR will build a sequence of trees where the next 397RT tries to improve the errors of the previous RTs. In its core concept, GBR builds an initial RT, calculates 398the errors of predictions with the initial RT and then builds a new RT to incrementally lower the prediction 399error and thus improve the final prediction model (Hastie et al., 2009). Several frameworks have been 400developed for implementing the boosting algorithm, including XGBR. For more technical details on XGBR, 401see Chen & Guestrin (2016). 402Hyperparameter tuning is done with the Tree-structured Parzen Estimator (TPE), a Bayesian optimizer. This 403method is chosen since the tuning process is considerably faster and more efficient than a grid search. 404Furthermore, the outcome is superior to a random search of hyperparameters (Yang & Shami, 2020). Models 405are then trained via 10-fold cross-validation (CV) on the train set.406The evaluation metrics are calculated on both train and test set, so that an impression can be obtained about 407the degree of overfitting of the models. Considering there are many evaluation metrics (Steurer et al., 2021), 408a selection of metrics, which were also considered in the studied literature, is chosen. These metrics are 409Mean Absolute Error (MAE), Root Mean Square Error (RMSE) and Mean Average Percentage Error 410(MAPE) and are defined according to the following formulas:411𝑀𝐴𝐸=4121413𝑛∑414𝑛415𝑖= 1416|𝑦𝑖― 𝑓𝑖|,417(1)418𝑅𝑀𝑆𝐸=4191420𝑛∑421𝑛422𝑖= 1423(𝑦𝑖― 𝑓𝑖)2,424(2)425Page 9 of 25426International Journal of Housing Markets and Analysis427142824293430443154326433743484359436104371143812439134401444115442164431744418445194462044721448224492345024451254522645327454284552945630457314583245933460344613546236463374643846539466404674146842469434704447145472464734747448475494765047751478524795348054481554825648357484584855948660487