Transcription
STRUCTURED SCENARIO ANALYSIS FOR OP RISKHIGH QUALITY RISK EVALUATIONSINTERNAL AND STRICTLY CONFIDENTIAL: PLEASE DO NOT DISTRIBUTE, COPY OR DISSEMINATEAPRIL 2020
The Analytics Boutique (TAB) is a risk analytics software company that builds userfriendly analytics solutions to uplift the risk capability of your institutionEnable user friendly and transparent analyticalprocessesBring in industry standards and best practicesin analyticsWe believe that analyticsteams, rather than designingand developing code, shouldbe focused on value addedtasks, being assisted by userfriendly tools with full modelgovernance, integrity of dataflows between analyticalprocesses and mechanisedreport generationProvide full model governance with audit trail,user control and thorough reporting featuresMinimise model errors as a result of theelimination of manual processesReduce dependence on coding experts due toautomation of analytic processes and data flowDeliver full model validation featuresOur Analytics delivers the money value of riskallowing a “monetary value based” managementWe help organisations move from data to action1Analytics made friendly
We are well recognised in the GRC industry for our award wining offerings in theop risk management, measurement and stress testing space2016/17 industry award recognition with 5 awards and 5 awards in 2018/192018/192016/172018/192016/17Best operational risksolution2019By InsuranceERMBy Risk.Net (Risk Magazine)2Analytics made friendly
We are thought leaders in the risk industry and have made significantcontributions to the advancement of the GRC Analytics industry"I found the quantitative methods presented in “Operational Risk Capital Models” to be not only rigorous, but also understandable and actually useableand useful, which can be said of shockingly few books treating operational risk. Amidst a wasteland of operational risk management pie charts andunactionable and subjective heat maps, books like this are an oasis of practical, applied solutions for capital estimation and stress testing. If yourobjective is to directly and measurably mitigate and manage operational risk using scientifically defensible, objective methodology, as opposed to redamber-green traffic ‘analyses,’ the methods herein are the kind you need."J.D. Opdyke, GE CapitalRiskBooks, Incisive MediaThe Actuary Magazine, Society of ActuariesMarcelo Cruz, Morgan Stanley3Analytics made friendly
Award winning Structured Scenario Analysis represents a breakthroughaddressing most burning issues in judgment based risk assessmentsCognitive biasesmitigation Structured Scenario Analysis is designed to mitigation multiple biases: need for closure, herding or group thinking,confirmation biases, anchoring biases, authority biases and otherrisk measurement &mitigation jointly In Structured Scenario Analysis, risk mitigation is evaluated together with risk evaluation, using a scientific methodbased on calculating the money value of riskMoney value of risk By on-the-fly Monte Carlo simulation, it calculates the cost of assuming risks and compares it with the savings ofhedging/controlling such risk providing the NPV of mitigation actionsScientific validation ofexpert judgment Structured Scenario Analysis implements performance based expert judgment which allows to validate respondingexperts based on limited available informationEfficiency featuresand for engaging yourorganization It enables a workflow, email sending system, expert responding progress page, reminders, answers automatedaggregation, extensive reporting, and moreCorrelation approach Structured Scenario Analysis provides a solid cross-scenario correlation approach based on expert judgment Correlations are very transparent, intuitive and easy to justifyRobust and stablecapital calculationAnalytics available to1st line of defenceStrong governanceFully flexible Structured Scenario Analysis integrates different sources of data (ILD, ED and BEICFs) to compute a more stablecapital charge, adding information of the distribution tails, reducing the volatility of capital estimates Structured Scenario Analysis provides, to the first line of defence, the cost of risk, saving from mitigation and NPV ofaction plans required investing, encapsulating all modelling complexities thanks to AI algorithms User control, audit trail, roles and activities differentiated by user and other Flexible forms, user defined number of loss collection processes, indicators, configurable workflow, etc.4Analytics made friendly
Structured Scenario Analysis a comprehensive range of functionality that catersfor holistic and efficient scenario analysis process1. Flexiblestructuringof riskmanagementprocess6. Findingsform2. Tools forengaging yourorganization7. Analyticsfullyintegrated3. Scientificvalidation ofexpertjudgmentStructuredScenarioAnalysis8. Userdefinedreporting4. Activemanagementof cognitivebiases9. Audit trails& governance5. Mitigation:money valueof risk andNPV of actionplans10.Documentsrepository5Analytics made friendly
Structured Scenario Analysis phases the risk evaluation process for exhaustivelycollecting and processing the required data and mitigating biasesScenario planningScenario analysis developmentCapital estimates0. Scenario Planning1. Scenariodocumentation andworkflow structuring2. Scenario presentationworshop4. Risk and rewardanalysis for mitigation7. Capital estimatesScenario identificationScenario description,classification andlabellingScenario presentationfor aligning expertsPre and post mitigationrisk profile iscalculatedScenario analysis aremodelled indistributionsScenario voting andpriorizationQuestionnaire isdefined: multiplesections if required3. Estimation of lossesby individualNPV of mitigation planis calculatedMonte Carlo simulationprovides VaR andcapitalExperts answerindividually or inworkshop5. Cause drivencorrelationsAggregate differentlegal entities capitalOpen and closequestionnaireAggregation of scenariovotingRelevant ILD, ED, KRIsand more is includedfor referencesSensitivity analysisExperts determinesensitivities to commonRiskAllocate additivecapital to scenariosPrioritised scenariosare sent todevelopmentLoss estimatesquestions are defined:worst lossesExperts ranked on seedquestions performanceis determinedRisk correlationcalculation arecalculated8. Approve capitalnumbersBayesian network canbe defined for detailedmodellingIndividual answers areaggregated based onranking6. Approve scenarioresultsApprove finalsimulations and capitalnumbersOverride final results,review and approveGenerate capitalreportsScenario scheduling:dates, reminders,format, etc.6Analytics made friendly
1Flexible structuring of risk management processStructured Scenario Analysis permits to define any number of user definedscenarios as well as leveraging from our scenario libraryIT risk scenario libraryConduct risk scenario library7Analytics made friendly
1Flexible structuring of risk management processStructured Scenario Analysis provides flexible risk evaluations in workshops orindividual responses and a customized structuring of the evaluation workflowThe organization of each risk evaluation can be fully customised based on the risk type needs regarding controls, mitigationanalysis. It also includes efficiency features such as automatic reminders, alerts, emails, full reporting, etc.Automatic reminderssent to non-respondingIf the risk evaluation is organized in a workshop format, then TABsends an invitation to participants with all required informationIn a second step, the loss estimates can be organized for SMEs’ responsesindividually (or in a workshop), to avoid multiple issues such as groupthinking, influence of leaders, fear of looking unsmart and so onThe risk evaluation can structured to contain any of these elements as selected by the OpRisk manager: support data, dependencyfactors for correlations, loss estimates, mitigation analysis, Other (means additional tabs for more specific analysis), findings(regulatory, internal audit, etc.), reminders, etc.8Analytics made friendly
1Flexible structuring of risk management processStructured Scenario Analysis is designed on flexibility. It allows the user to defineas many questions, question types, quantification methods, number of tabs 9Analytics made friendly
1Flexible structuring of risk management processStructured Scenario Analysis allows to fully customize risk evaluation formsincluding the number and content of tabs, from very simple forms to lengthy onesIn the below example for conduct risk management a questionnaire has been created with tabs regarding “Support data”,“Dependency factors”, “Loss estimates” and so onThe next example TAB tool has been configured for model risk and governance and includes tabs regarding the model lifecycle: “Development”, “Validation”, “Approval”, “Audit” and so on10Analytics made friendly
2Tools to engage your organizationTAB provides multiple features for engaging your organization in obtaining qualityand timely responses when evaluating scenario analysisTAB permits a dynamic and flexible invitation process via email to include participants into the op and conduct risk evaluationprocess. TAB sends invitations via email to selected participants in risk evaluation including .ICS file (for the calendar),customised message and name of the rest of participants11Analytics made friendly
2Tools to engage your organizationTAB provides a facility to review participants risk evaluation progress and permitsto send them feedback emails for optimizing responses quality and response timeInclude a .ICS file regarding a potential meeting with SMEs, including time, date, duration and locationWrite email text to selected SMEsThe progress of individual SMEs can bereviewed in the actual form, approved, furtherinformation can be requested and so onMultiple SMEs are requested to evaluate the same scenarioindividually to avoid known cognitive biasesThe degree of progress of the different SMEs inresponding to their questionnaire is easily trackedSelect SMEs to send be sent email12Analytics made friendly
2Tools to engage your organizationStructured Scenario Analysis is full of proactive features: i.e. automatedreminders for guaranteeing a timely response and actions from the organizationThe tool permits to create reminders for specific tasks in risk maintenance and managementTasks can be directed to specific stakeholders in the risk management chainThese may include regular tasks as periodic re-evaluation, identification of new emerging risks but also specific tasks as e.g. mitigation actions from validations to the responsible stakeholders, 13Analytics made friendly
2Tools to engage your organizationIt provides a chatting facility between the scenario manager and the subjectmatter expert, for expediting and maximizing the quality of answers obtained14Analytics made friendly
3Scientific validation of expert judgmentSeed questions can be embedded into the questionnaire for assessing the expertsskills in evaluating uncertain risk events and to weight their answers accordinglySeed questions are questions included into the questionnaire to exam the skills of participating experts. The performance onin these seed questions is used to score the skills/knowledge of experts in predicting uncertain risk events. Such score is usedto give more weight to best performing experts when aggregating answersDefinition of seed questions into the scenarioquestionnaire15Analytics made friendly
3Scientific validation of expert judgmentThe expert answers are aggregated based on their seed questions performancescore and such aggregated answer is used for risk evaluation and modellingScenario analysis evaluations from multiple experts are aggregated to a single answer per Risk, based on the performance ofeach expert on the seed questions. Best performing experts are given more weight in the aggregated answerSME answers are aggregated based on seed questions performanceThe weight from the seed question performance can be overridden16Analytics made friendly
4Active management of cognitive biasesScenario analysis relies on expert judgment which needs the mitigation ofcognitive biases to guarantee a minimum quality of assessmentsBiases with strong influence inexpert based assessmentsNeed for closure biasStructured Scenario Analysis features to mitigate biases Risk scenario identification modules: an open question questionnaire to identify the list of largest risk concernsamong selected experts and a close risk list questionnaire for voting and prioritizing risk scenariosHerding or group thinking bias Permits answering scenario questionnaire in a workshop or each expert individually, to avoid group thinking Answering in various steps: risk description workshop, experts individual answers, risk mitigationAnchoring bias The scenario analysis questionnaire contains a section to relevant internal and external losses and other riskmanagement metrics to mitigate SME s inadequate anchors and pre-existing referencesDenial bias Links risk evaluation and risk mitigation permitting to justify the losses and errors by proposing remediation andcalculating the NPV of action plans, to facilitate a risk exhaustive identification and proper evaluationConfirmation bias In a first phase, before answering individually loss estimates, a workshop analysis of scenario causes andconsequences is performed to eliminate pre-existing believes and establish a common understandingAuthority biasNatural and/or formal leadersFear of looking unknowledgeable Individual answering of risk evaluation questionnaire (as an alternative to workshop) to avoid the influence offormal or natural leaders and mitigate the fear of looking unknowledgeable in front of peers and superiors Possibility of structuring the scenario workshop in phases: 1) Joint workshop to present the risk and support data2) Individual answering of loss estimates 3) Individual/workshop answering of mitigation 4) answer aggregationLack of involvement Dissimilar degree of skills amongexperts Seed questions (a question whose answer is known) to examine the expert skills in evaluating uncertain riskscenarios permitting to identify those experts with better risk prediction skillsLinking risk evaluation and risk mitigation to motivate participating expertsPrevalence of intuition vsanalysis Enforcing robust analysis: analysis of the causal pathway, risk drivers, dependency factors and moreCreating the appropriate references by providing the internal and external losses and other OpRisk metricsSense of lack of usefulness Linking risk evaluation with risk mitigation and the evaluation of the NPV of action plansPermits to justify the business case of mitigation and insurance, by the NPV of required investment/costsLack of correlation approach Factor model correlation approach driven by SME judgment elicitation provides a solid and transparentcorrelation matrix, for scenario analysis capital modelling17Analytics made friendly
4Active management of cognitive biasesTAB allows to present supporting data such as ILD, ED and BEICFs, to help expertsin analysing risks and help to mitigate anchoring, recency and other biasesWithin the scenario analysis questionnaire, it is possible to include support data to help experts in their evaluations greatlyimproving the evaluation quality, such as internal and external loss data, KRIs, KCIs, KPIs, case studies Internal and external data relevant to the risk evaluationRCSA relevant to the risk scenario18Analytics made friendly
5Money value of risk and NPV of mitigation actionsTAB’s quantification methodologies permit to translate GRC metrics into themoney value of risk easily integrating such metrics into the daily managementTAB s differential quantification scopeRisk evaluationRisk odelRisk andcontrol rioITBrandBusinessIndicators:KRI, KPI,KPC, Action planmanagementExternal lossdata storageEconomic value basedmanagementLoss modellingInternal and externalCapital requirements andallocationScenario analysismodellingRisk appetite metricsHybrid model: ILD, ED, SAand Bayesian NetworksMitigation plans, insurance& controls NPVCorrelationsExpert elicitatedLoss forecasting and stresstestingVaROpVaR, Brand VaR, IT VaRReporting incl. regulatoryGRC Analytics scientificvalidationIntegration in financialplanning and strategyInternal lossdatacollection19Analytics made friendly
5Money value of risk and NPV of mitigation actionsTAB permits an on-the-fly Monte Carlo simulation of risk for evaluating the riskprofile before and after mitigation and determine the effectiveness of controlsTAB enables Analytics to the first line of defence to evaluate the risk profile and the impact of mitigation, using an on-the-flyMonte Carlo simulation feature. This helps to change the internal culture of the institution towards scientific risk evaluationand robust risk mitigation decisioningPre-mitigation risk profilePost-mitigation risk profile20Analytics made friendly
5Money value of risk and NPV of mitigation actionsThe mitigation plan Net Present Value (NPV) can be calculated in order to buildthe business case for justifying implementation and required investmentsTAB enables via its web application a feature to calculate the NPV of mitigation actions. The differences in the risk profile ofbefore and after mitigation together with the implementation cost of the control permits to determine whether the actionplan add or not net present value to the institution21Analytics made friendly
5Money value of risk and NPV of mitigation actionsThe impact of mitigation plans and insurance programs on the scenario riskprofile can also be evaluated and its NPV calculatedNPV and loss analysis applying insurance onlyNPV and loss analysis with insurance in the mitigation plan22Analytics made friendly
5Money value of risk and NPV of mitigation actionsSSA permits to model scenarios using a variety of methods including Bayesiannetworks for those scenarios most sensitive to current exposures or in whichdetailed analysis is needed for a precise estimation of losses or mitigation impactA complete and efficient modelling of scenario analysis requires the combination of modelling methods. Bayesian networks maybe used in exposure sensitive scenarios or requiring a precise loss estimation or mitigation NPV. Less critical scenarios might bemodelled using direct and less resource intensive methods such as direct estimation of losses (worst loss in 10 years )Pre-mitigation analysisPost-mitigation analysis23Analytics made friendly
6Findings, controls & action plan managementStructured Scenario Analysis offers action plans management, findings registersand controls for an effective managementRisk measurement would be an Academic exercise if there are no decisive actions towards Risk management/mitigation. Oursolution is built around such believe and provides multiple means to evaluate the effectiveness of controls and themanagement of mitigation actionsFindings forms24Analytics made friendly
7Analytics fully integratedSSA provides the means to determine correlations across risk scenarios through aconfigurable factor model elicited by SME judgment SMEs provide their estimate on the influence of the different external environment factors that impact crystallisation ofrisks. These estimates are later weighted by the seed questions performance score obtained by each SME The final risk scenarios correlation matrix is calculated with the correlations across the dependency factors and the weightof each factor in the risk scenarioDefinition of risk factor to guide SMEsSMEs provide their estimates on the influence of the risk drivers in the scenarioRisk factor name25Analytics made friendly
7Analytics fully integratedFinally, through the use of scenario modeling and Monte Carlo simulation, SSAproduces capital estimates and other risk management metricsFinally, capital is allocated following multiple methodologies (Euler Allocation, contribution to expected shortfall, etc.). Themodule for capital reporting permits the simultaneous representation of multiple runs performed at different time, differentfeatures, etc.Graphical representation of capital allocationCapital allocatedSolvency standards26Analytics made friendly
7Analytics fully integratedOur GRC Analytics are flexible and can be applied across other GRC Riskcategories such as BrandRisk and obtain Brand VaR or IT Risk27Analytics made friendly
8User defined reportingStructured Scenario Analysis is able to provide user defined reports anddashboards in an easy and intuitive way so that the organisation can track bothmodel risk and efficiency across the model lifecycleReporting functionalityUser-defined chartsUpdates dynamically as newdata is addedDrag and drop configuration of chartsAbility to have multiple dashboards fordifferent user-typesReporting on the model process andefficiency – time taken to develop, validateand approve models, costs per model etcMultiple chart / tableoptions28No programming requiredAbility to download toExcelTrack model risk andmodel efficiencyAnalytics made friendly
8User defined reportingRisk management and measurement data is stored in Structured Scenario Analysisdatabase and can be reported as desired using pivot tables and dashboardsInformation is captured by1 users or produced by GRCAnalytics forms and storedin a database2Pull information from database and createtables and charts as required usingintegrated reporting module3Graphs aregrouped intodashboards4Multipledashboardscan becreatedMultiple optionsavailable forrepresentation29Analytics made friendly
9Full audit trailExtensive information about which user activity gained accessed / created /altered workflows and their components provides comfort for audit andgovernance structuresGeneral activity log including all scenarios and usersActivity log specific to one scenario30Analytics made friendly
10Document inventoryAll documents used or captured such as case studies or attachment to answersare consolidated into a document repository reportOur solution creates a document inventory with all documents used for providing an answer or the case studies attached tofroms for informing expert answers31Analytics made friendly
32Thank you.The information contained herein is proprietary, confidential and may be legally privileged. Please do not distribute this presentation withoutthe prior written consent of The Analytics Boutique Ltd or its authorised affiliates. 2019 The Analytics Boutique Ltd32Analytics made friendly
Structured Scenario Analysis is designed to mitigation multiple biases: need for closure, herding or group thinking, confirmation biases, anchoring biases, authority biases and other risk measurement & mitigation jointly In Structured Scenario Analysis, risk mitigation is evaluated together with risk evaluation, using a scientific method
