WIXLIB

Model RiskManagementNav VaidhyanathanDirector, Head of Model Risk ManagementWintrust Financial CorporationGlobal Association of Risk ProfessionalsAugust 2014

These materials reflect the views and opinions of the speaker and are notintended to reflect the views, policies or practices of Wintrust Financial or itsaffiliated companies. They are for informational purposes only.The views expressed in the following material are the author’s and do notnecessarily represent the views of the Global Association of RiskProfessionals (GARP), its Membership or its Management.2

AgendaModel Risk Management is a broad, evolving, and an expanding topic. This presentationwill cover only select components at a high level. Inherent risk (model tier) and residual risk of model A framework for model monitoring Model risk aggregation framework Model validation for stress test (e.g., DFAST) models3 2014 Global Association of Risk Professionals. All rights reserved.

Model Risk Axioms*The following can potentially be considered as self-evident truths for Model Risk. Giveneverything else equal, All models have limitations Model Risk is related to model limitations Model Risk increases with more models (this might fall in the grey area) E.g., two Tier 1 models potentially pose higher risk than 1 Tier 1 model (everything else being thesame, i.e., similar limitations, similar validation status, similar materiality, similar findings, etc.) Model Risk for Tier 1 model Model Risk for Tier 2 model Model Risk for validated model Model Risk for unvalidated model (for the same model) More findings more Model Risk (same category and level of findings) Certain types of findings will have higher Model Risk than others Models with deteriorating performance pose higher Model Risk Model Risk is exhibited by the variance between model output and observed actual(this is just one of the ways that Model Risk exhibits itself)* may not be axioms in the purest sense of the term4 2014 Global Association of Risk Professionals. All rights reserved.

Residual Risk Rating could bedriven by: Model validation findings: Category Level # of Open Findings Model performance /monitoringResidualRisk RatingInherent Risk and Residual Risk of ModelModel owners can reduce residualrisk – by improving models,improving controls, and/orremediating findingsHighModerateLowLow (Tier 3)ModerateInherent Risk Rating (Model Tier)Inherent Risk Rating could be driven by: Materiality Type of model Used for regulatory / financial reporting or critical businessdecisions Feeds into another Tier 1 model?Model owners cannot (usually) reduce inherent risk5 2014 Global Association of Risk Professionals. All rights reserved.High (Tier 1)

Residual Risk Rating Framework Residual Risk Rating (High,Moderate, Low) is based onthe Residual Risk Score range E.g., RRS 200 High, RRS 100 Low If the model is deemed unfit foruse, the RRR is High The Residual Risk ScoreRange to Residual RiskRatings mapping needs to bemonitored periodically The ranges can be determinedby typical observations frommodel validation and modelmonitoring exercises If a model is unvalidated, theresidual risk rating is HIGH6 2014 Global Association of Risk Professionals. All rights reserved.

Model Monitoring Model monitoring should evaluate whether changes (including anticipated) inproducts, exposures, activities, clients, or market conditions necessitateadjustment, redevelopment, or replacement of the model Ongoing activities include – review of input data and assumptions, outcomeanalysis, model stability checks, review of overlays and expert judgments,benchmarking, etc. Key considerations Model performance monitoring›Pros and cons of model performance measures›Model type›Model use›Type of measure Periodicity of monitoring Thresholds for model recalibration or rebuild From performance measure to model performance rating7 2014 Global Association of Risk Professionals. All rights reserved.

A Bottom Up Framework For Model Risk AggregationImpact of Failure (e.g., on Capital)YesHigh ResidualRiskThe end nodes in thedecision tree can beused for aggregatingmodel riskTier 1Tier 2Tier 3Validated NoModerateResidual RiskLow ResidualRiskAs part of an RCSAprocess, the impact ofthe model failure isestimated by modelownersFor DFAST projection models, for Severely Adverse Scenario, the impact on capital can be assessedusing backtesting during Great Recession period for all models (adjust the variance by severity of thescenario compared to the Great Recession)By looking at the distribution, a conservative starting point for capital can be chosen for each modelTier8 2014 Global Association of Risk Professionals. All rights reserved.

Some Considerations in Validation of Stress Test Models Are estimates conditioned on macroeconomic scenarios? Do the models produce outcomes that show separation and order between the baseline,adverse, and severely adverse scenarios? Are the scenario and assumptions consistent across all models? Are exposures modeled using appropriate levels of segmentation? Qualitative elements of the model Relevance of third party vendor models to company specific characteristics, especially whenusing proxy portfolio Sensitivity analysis around assumptions Use of weak models vs. expert judgment For models not validated, compensating controls and conservativeness Documentation Capital buffer for model risk9 2014 Global Association of Risk Professionals. All rights reserved.

C r e a t i n g a c u l t u r er i s k a w a r e n e s s o fGlobal Association ofRisk Professionals111 Town Square Place14th FloorJersey City, New Jersey 07310U.S.A. 1 201.719.72102nd FloorBengal Wing9A Devonshire SquareLondon, EC2M 4YNU.K. 44 (0) 20 7397 9630www.garp.orgAbout GARP The Global Association of Risk Professionals (GARP) is a not-for-profit global membership organization dedicated to preparing professionals and organizations to makebetter informed risk decisions. Membership represents over 150,000 risk management practitioners and researchers from banks, investment management firms, government agencies,academic institutions, and corporations from more than 195 countries and territories. GARP administers the Financial Risk Manager (FRM ) and the Energy Risk Professional (ERP )Exams; certifications recognized by risk professionals worldwide. GARP also helps advance the role of risk management via comprehensive professional education and training forprofessionals of all levels. www.garp.org.10 2014 Global Association of Risk Professionals. All rights reserved.

Model RiskManagement forNon BanksMichelle McCarthyManaging Director, Risk ManagementNuveen InvestmentsGlobal Association of Risk ProfessionalsAugust 2014

DisclaimersThese materials reflect the views and opinions of the speaker and arenot intended to reflect the views, policies or practices of NuveenInvestments or its affiliated companies. They are not intended toprovide investment advice and are for informational purposes only.The views expressed in the following material are the author’s anddo not necessarily represent the views of the Global Association ofRisk Professionals (GARP), its Membership or its Management.2

Contents Differences between models and their uses at banks vs. the buy side What should be in scope for model risk management activity in the assetmanagement business? Potential tiering of asset management models, and effect on model riskmanagement activities3 2014 Global Association of Risk Professionals. All rights reserved.

Most literature on model risk management has concerned banks OCC 2000-16 OCC 2011-12, FRB SR 11-7 The case of AXA Rosenberg brought the conversation to the buy side, withoutmuch guidance on how bank practices can and should differ from asset managers’practices4 2014 Global Association of Risk Professionals. All rights reserved.

Different uses of models in asset managers vs. banks Asset managers are not computing regulatory capital Bank-owned asset managers may need to provide operational risk capitalcalculations, but this is not in service of the asset management business We rarely use valuation models as the sole source of valuation; we frequentlyuse external sources Many asset management models are an input to a decision, but not the soledriver of that decision5 2014 Global Association of Risk Professionals. All rights reserved.

Potential scope of model management activity for asset managers Includes models that are used for investing or hedging Drive investing activity to some extent Value key assets Does not include calculators; model must provide estimates and involveuncertainty to require model risk management Good change controls always an important operational control, but calibrations andvalidations should be reserved for models that estimate uncertainty Would not include compliance systems Would not include price-to-yield calculations Would not include simple screens for fundamental stock characteristics Would not include pure index replication models without meaningful uncertainty Does not include analyses that help develop an investment thesis—only thetools used to execute the thesis, or to value assets once purchased66 2014 Global Association of Risk Professionals. All rights reserved.

Tiering for materiality A potential tiering of asset management models:Tier 1Tier 2Models that: have high potential directimpact on company results are broadly applied across theentire company are understood andmaintained by a relativelynarrow range of experts at thecompany Result in instant electronictrading with little or no abilityto intervene, and/or affectinga large portion of the portfolio Such as: Statistical arbitrage model High frequency trading model A model that is rigorouslyadhered to, is key to a broadset of strategies and isdisclosed as a critical featureof the investment process Significant valuation modelswhere third party sources areunavailable Tier 3are tightly coupled to trading but permithuman intervention, and/or affect asmall portion of the businessresult in trading with little other inputthan direction from a model (quantstrategies)provide hedge ratios for usingderivatives, or to balance proportionsof fixed income instruments, inportfoliosvalue smaller, less significant balancesheet items than those captured in Tier1 A strategy that strongly adheres torebalancing if Barra or Yieldbookmeasures exceed a thresholdAn economic indicator model that isthe basis of an investment strategyHigh frequency trading modelsModels used for delta hedging or assetallocation rebalancingMinor valuation models where thirdparty sources are unavailable 7 2014 Global Association of Risk Professionals. All rights reserved. Tier 4apply a screen to a universe ofstocks, using some measure ofuncertainty, such as a measurementof the dispersion of somefundamental characteristic (low priceto-earnings ratios, for example)compare characteristics of securitiesto assist the portfolio manager infinding securities that he or shedeems desirable Using Barra or Yieldbook to seewhere a portfolio is overweight, inorder to select trades that will balancethe portfolio betterEconomic indicator screens toprovide general investment signalsIndex replication models with tiltsRisk oversight modelsValuation models where third partyvaluations are available may support a certainamount of decisionmaking, but with manyother inputs to thesedecisions outside themodel.are outside thecompany’s areas of keybusiness riskare understood by abroad range of businesspeopleMeasures of portfoliosthat are not used todrive actionIndex replicationmodels that approach100% replication