WIXLIB

Survey Research - AdvantagesOne of the only ways to measure “non-use” values– Environmental impairment– Both Goods (willingness to pay) and Bads(willingness to accept)– Private impacts of public goods (airports, rails-totrails, etc.)

Survey Research - AdvantagesOne of the only ways to measure “non-use” values– Environmental impairment– Both Goods (willingness to pay) and Bads(willingness to accept)– Private impacts of public goods (airports, rails-totrails, etc.)Often referred to as “passive use” values

Survey Research - AdvantagesEnormously flexibleWidely used both in appraisal and other areas – welldeveloped body of knowledgeMethodology outlined in the Reference Manual onScientific Evidence published by the U.S. JusticeDepartmentResults easy to analyze and describe –understandable and compelling to a Court

Survey Research - Challenges1993 NOAA Panel – questions which have beenansweredConfusing WTP with WTA(Note: WTP has an implicit income constraint)Confusing cross-sectional and longitudinal studies

Survey Research – “Must Read”Diamond, “Survey Research”, in Reference Manualon Scientific Evidence (U.S. DOJ, 2010)Mundy and McLean, “Addition of CV to the RequiredBody of Knowledge”, in J. Real Estate Practice andEducation (ARES, 1998)Lipscomb, Mooney, and Kilpatrick, “Do CV ResultsSystematically Differ from Hedonic RegressionResults? Evidence From a Residential PropertyMeta-Analysis”, J. Real Estate Literature, 2013

Meta AnalysisQuestion – how do we statistically compile thecollected wisdom of scholars in the field?

Meta AnalysisContrasting and combining results from differentstudies,Identifying patterns among study resultsIdentifying sources of disagreementIdentifying other interesting relationships

Meta AnalysisOne recent use -- to assess diminution in valueresulting from environmental disamenities

Meta AnalysisAssume we have three published studies aboutgroundwater contamination:Study# obs% dim12330152525%40%45%

Meta AnalysisWhat if we took a weighted average of these studies?Study# obs130215325Total70average of 3 studies% dim Weighted25%40%45%11%9%16%35%

Meta AnalysisIn reality, meta analyses look at dozens of studies,and control for such factors as type of contaminant(groundwater, air, etc.), geographic region,neighborhood type (rural, suburban, etc.), timefactors, and underlying unimpaired property value.

Meta AnalysisIn reality, meta analyses look at dozens of studies,and control for such factors as type of contaminant(groundwater, air, etc.), geographic region,neighborhood type (rural, suburban, etc.), timefactors, and underlying unimpaired property value.The exact control variables are less important thanarriving at a good explanatory “fit”

25%The “file drawerproblem” with metaanalyses -- acautionary taleBenfit from Transit AccessPercentage Benefit20%15%10%5%0%051015Study Number2025

25%The “file drawerproblem” with metaanalyses -- acautionary taleBenfit from Transit AccessPercentage Benefit20%15%Imagine we’reconducting a metaanalysis, and theseare the 20 studieswe find published10%5%0%051015Study Number2025

25%The “file drawerproblem” with metaanalyses -- acautionary taleBenfit from Transit AccessPercentage Benefit20%15%What if there wereactually 30 studies,but 10 didn’t getpublished?10%5%0%05101520Study Number253035

25%The “file drawerproblem” with metaanalyses -- acautionary taleBenfit from Transit AccessFortunately, this is rarely a problemin real estate studies – negative orzero findings are usually just as15%interesting and publishable aspositive ones!Percentage Benefit20%What if there wereactually 30 studies,but 10 didn’t getpublished?10%5%0%05101520Study Number253035

Meta AnalysisUsefulness?– Contaminated Property Litigation– Housing Characteristics– Real Estate Brokerage Effectiveness– REIT & RE Portfolio Valuation– Appraisal Accuracy– Environmental Amenities– Transportation Infrastructure & Airport Noise– Historic Preservation

Meta Analysis – “must read”Simons & Saginor, J. Real Estate Research, 2006Lipscomb, Mooney, & Kilpatrick, J. Real EstateLiterature, forthcoming, 2013Sirmans, MacDonald, Macpherson, & Zietz, NationalCenter for R.E. Research, 2005Debrezion, Pels, & Rietveld, J. Real Estate Finance &Economics, 2007Nelson, J Transport Economics and Policy, 2007

Expert SystemsWhat happens “in between” the hedonic model andthe sales adjustment grid?Is there a way to bring the power of statisticalanalysis to smaller data sets or “dirty” data?How can we incorporate the appraiser’s “judgment”into statistical processes?

Expert SystemsTies together several streams of research:––––StatisticsAppraisal Theory and PracticeSet TheoryEquilibrium Theory

Expert SystemsBut first a word about WLS – weighted least squaresOLS (ordinary least squares) only works if the errorterms (the “residuals”) are normally distributed witha mean of zero and a standard deviation of 1What if the error terms are badly behaved?

Expert SystemsWhat could cause badly behaved errors?Spatial colinearityTemporal colinearityNon-negative variables (i.e. – truncated prices)

Expert SystemsWhat could cause badly behaved errors?Spatial colinearityTemporal colinearityNon-negative variables (i.e. – truncated prices)These are all common in real estate data

Expert SystemsIn a pure regression model, we accommodate thiswith what’s called “weighted least squares”Some uniform weighting is appliedExample – using a logarithmic adjustment in hedonicpricing models

Expert SystemsHence, when we apply regression or other statisticalmodeling to the appraisal process, one challenge isto incorporate appraiser judgment to adjust for“non-normality” in the data.Recall Colwell, et al. TAJ 2009 – both appraisal andstatistics training are necessary in modeling

Expert Systems:The Bayesian EstimatorPr(E H ) Pr(H )Pr H E Pr(E )

Expert Systems:The Maximum LikelihoodEstimator (MLE)Given what we know about this data, what probabilitymodel fits best?

Expert Systems:The Maximum LikelihoodEstimator (MLE)Consistency: The MLE converges asymptotically to thevalue being estimated. From an appraisalperspective, this means that there is a benefit toexperience and professionally developed judgment.

Expert Systems:The Maximum LikelihoodEstimator (MLE)What do we mean by “asymptotic”?As “N” gets larger and larger, the equation becomesmore and more normal.The Central Limit Theorem is an example of anasymptotic distribution

Expert Systems:The Maximum LikelihoodEstimator (MLE)Asymptotic normality: As sample size increases, theMLE distribution tends toward a normal distribution.

Expert Systems:The Maximum LikelihoodEstimator (MLE)Asymptotic normality: As sample size increases, theMLE distribution tends toward a normal distribution.f ( x) N (as n )

Expert Systems:The Maximum LikelihoodEstimator (MLE)Efficiency: There is no asymptotically unbiasedestimator that has lower asymptotic mean squareerror.

Expert Systems:The Maximum LikelihoodEstimator (MLE)So why don’t we use MLE more often?

Expert Systems:The Maximum LikelihoodEstimator (MLE)So why don’t we use MLE more often?nln L( x1.xn ) f ( x1.xn ) ln f ( xi )i 1

Expert SystemsTies together several streams of research –Appraisal Theory and PracticeShiller and Weiss (1999) – Type II errorsMass AppraisalDaubert & Kumho Tire

Expert Systems

Expert SystemsShiller & Weiss – Type II errorscause appraisal models tounderestimate risk

Expert SystemsA brief sojourn into DaubertDoes the method center upon a testable hypothesis?Is there a known or potentially knowable error rateassociated with the method?Has the method been subject to peer review?Is the method generally accepted in the relevantscientific community?

Expert SystemsA brief sojourn into DaubertDo(es) the method(s) center upon a testable hypothesis?Is there a known or potentially knowable error rateassociated with the method(s)?Has the method been subject to peer review?Is the method generally accepted in the relevantscientific community?

Expert SystemsTie together several streams of research –Set TheoryFuzzy Logic(as distinct from neural networks)

Expert SystemsTie together several streams of research –“ a ‘fuzzy set’ extends the concept of membership in aset to situations in which there are many, possibly acontinuum, of grades of membership.”-- Zadeh (1964)

Expert SystemsTie together several streams of research –“ a ‘fuzzy set’ extends the concept of membership in aset to situations in which there are many, possibly acontinuum, of grades of membership.”From a statistician’s perspective,this sounds a whole lot like theway we determine “comparables”-- Zadeh (1964)

Expert SystemsTie together several streams of research –Equilibrium TheoryWhat is the nature of the real estate market?Nash (1951)

Expert SystemsTie together several streams of research –Equilibrium TheoryWhat is the nature of the real estate market?Nash (1951)Why does this matter?

Expert SystemsTie together several streams of research –Equilibrium TheoryWhat is the nature of the real estate market?Nash (1951)Why does this matter?(hint: modeling heterogeneity)

Expert SystemsThe real estate transactional market constitutes a Nashequilibrium, in which all participants take into accountthe strategies of other participants in the goal ofoptimizing their utility.

Expert SystemsThe real estate transactional market constitutes a Nashequilibrium, in which all participants take into accountthe strategies of other participants in the goal ofoptimizing their utility.Thus, spatial and temporal autocorrelation are part of theprocess, rather than aberrations to the model.

Expert SystemsThe appraiser, faced with a set of data and a set of priorobservations about the underlying market, uses fuzzylogic to formulate a maximum likelihood estimator todetermine the true value of the property.

Expert SystemsThe appraiser, faced with a set of data and a set of priorobservations about the underlying market, uses fuzzylogic to formulates a maximum likelihood estimator todetermine the true value of the property.S/he is able to do this with a limited data set based onthe Bayesian priors already known about theprobable behavior of the market.

Expert SystemsAdapted fromMcClusky & Anaud’sFigure 4

Expert SystemsTwo Case Studies: Plaquemines Parish, LA Lomax, IL

Expert SystemsTwo Case Studies: Plaquemines Parish, LA Lomax, ILCOD abs(Price Value )median valuen

Expert SystemsCOD COD:Coefficient ofDispersionFrom the IAAOSupplementalStandards abs(Price Value )median valuen

Expert SystemsCOD COD:Coefficient ofDispersionFrom the IAAOSupplementalStandards abs(Price Value )median valueA measurement ofthe accuracy of thestatisticn

Expert SystemsParametric Statistics – used when we expect that theunderlying data is well distributed (i.e. – normal)Non-Parametric Statistics – used when the data is notexpected to be well defined or well distributed

Expert SystemsParametric Statistics – used when we expect that theunderlying data is well distributed (i.e. – normal)Example – Mean, Standard DeviationNon-Parametric Statistics – used when the data is notexpected to be well defined or well distributedExample – Median, Coefficient of Dispersion

Expert SystemsTwo Case Studies: Plaquemines Parish, LA Lomax, ILCOD abs(Price Value )median valuen

Plaquemines Parish, LA

Shortly after Katrina

Proposed PlaqueminesParish Class Action Area

Proposed PlaqueminesParish Class Action AreaNote Sample Area

Table 1Case Study No. 1 Valuation ExampleProperty TypeBase Value Qual/Cond BrickAcresSFDistanceCoefficients 20,000 5,000 3,000 25- 500Property type “J” base values 48,000200.5120017Property type “I” characteristics210.3135021Vali 48,000 ((2-2)* 20,000) ((1-0)* 5,000) ((0.3-0.5)* 3,000) ((1350-1200)*25) ((21-17)*-500)Vali 48,000 0 5,000 - 600 3,750 - 2,000Vali 54,150C.O.D. 9.06%

Proposed Lomax, IllMass Tort Area

Table 4Lomax Coefficients of DispersionProperty TypeDouble-WideMobile HomeOld Style 1Old Style 2RanchC.O.D8.05%35.21%21.81%13.40%7.67%

Lessons Learned & Avenues for Future ResearchThe use of MLEs in an expert systemThe use of non-parametric models to evaluate statisticalproperties (“known” error rates)How is this applicable in a single-property appraisal model?

Kendall’s tau (for collinearity & dependence)-- also Spearman’s ρKolmogorov-Smirnov (for normality)Mann-Whitney (to test if the difference in mediansbetween two populations equals zero)-- also Χ2Kruskal-Wallis statistic (variance)

Expert Systems – “must read”Kilpatrick, “Expert Systems and Mass Appraisal”, J.Property Investment and Finance, 2011Lentz and Wang, J. Real Estate Research, 1998 – tellsus that the sales adjustment grid is just a special caseof the weighted least squares regression hedonicmodel.

USPAP?Scope of Work Rule – current wording:“The scope of work is acceptable when it meets orexceeds: The expectations of parties who are regularlyintended users for similar assignments; andWhat an appraiser’s peers’ actions would be inperforming the same or a similar assignment”

USPAP?Scope of Work Rule – Proposed 2014 update:“The scope of work is acceptable when it meets orexceeds: Both the expectations of parties who are regularlyintended users for similar assignments and what anappraiser’s peers’ actions would be in performing thesame or a similar assignment; or

USPAP?Scope of Work Rule – Proposed 2014 update: Recognized methods and techniques prescribed bythe Appraisal Practices Board of the AppraisalFoundation, or in other peer-reviewed, publishedappraisal or valuation books and articles, andpublished appraisal or valuation coursework taught bya college, university, professional appraisal orvaluation organization, or state and federalgovernment agencies.”

USPAP?Scope of Work Rule – Proposed 2014 update: Recognized methods and techniques prescribed bythe Appraisal Practices Board of the AppraisalFoundation, or in other peer-reviewed, publishedappraisal or valuation books and articles, andpublished appraisal or valuation coursework taught bya college, university, professional appraisal orvaluation organization, or state and federalgovernment agencies.”

USPAP?Standards Rule 1-1 (a)“In developing a real property appraisal, an appraisermust be aware of, understand, and correctlyemploy those recognized methods and techniquesthat are necessary to produce a credible appraisal.”

USPAP?Standards Rule 2-2(a)(b)(c) (viii)“(Describe) (Summarize) (State) the informationanalyzed, the appraisal methods and techniquesemployed, and the reasoning that supports theanalyses, opinions, and conclusions .”

USPAP?Advisory Opinion 18“An AVM’s output is not, by itself, an appraisal, andcommunication of an AVM’s output is not, in itself, anappraisal report.”

USPAP?Advisory Opinion 18“An AVM’s output is not, by itself, an appraisal, andcommunication of an AVM’s output is not, in itself, anappraisal report.” Both USPAP 1 and 6 may apply

USPAP?Standards Rule 6-4(b) comment:“Mass appraisers must develop mathematical modelsthat, with reasonable accuracy, represent therelationship between property value and supply anddemand factors, such as presented by quantitativeand qualitative property characteristics.”

USPAP?Standards Rule 6-4(c) comment:“Models must be calibrated using recognized techniques,including, but not limited to, multiple linear regression,nonlinear regression, and adaptive estimation.”

USPAP?Comment to Standards Rule 3-1(a)Changes and developments in economics, finance, law, and societycan have a substantial impact on the appraisal profession. Tokeep abreast of these changes and developments, the appraisalprofession is constantly reviewing and revising appraisalmethods and techniques and devising new methods andtechniques to meet new circumstances. Each appraiser mustcontinuously improve his or her skills to remain proficient inappraisal review.

USPAP?Comment to Standards Rule 3-1(a)The reviewer must have the knowledge and experience needed toidentify and perform the scope of work necessary to producecredible appraisal assignment results. Aspects of competencyfor an appraisal review, depending on the review assignment’sscope of work, may include, without limitation, familiarly with thespecific type of property or asset, market, geographic area,analytical methods, and applicable laws, regulations, andguidelines.

CPE?E.R. 1-4It is unethical in the performance of a service toknowingly fail to:(a) Identify the appropriate Standards to be applied(b) Disclose in any report the Standards applied(c) Take all steps necessary or appropriate to understandthe Standards applied

CPE?E.R. 1-4It is unethical in the performance of a service toknowingly fail to:(a) Identify the appropriate Standards to be applied(b) Disclose in any report the Standards applied(c) Take all steps necessary or appropriate to understandthe Standards applied

Dr. John Kilpatrick, MAI, is an appraiser and financial economist and is the CEO of Greenfield Advisors,specializing in economic market and valuation analysis, principally in real estate matters, withoffices in Seattle and Atlanta. He also serves as a Visiting Scholar in Real Estate Finance atBaruch College, City University of New York, and is a nationally certified USPAP instructor.His Ph.D. i

Advanced Statistical Modeling In Real Estate Appraisal John A. Kilpatrick, Ph.D., MAI August 2, 2012 www.greenfieldadvisors.com Suite 240 – 2101 4th Ave Suite 1000 - 1870 Th