WIXLIB

How to Address the Risk Tolerance Problem Address the organization, processes, andthe physical system Select program/data quality measuresand periodically evaluate them Quantify the causal relationship of rootcauses of each threat Incorporate the historical data, subjectmatter expert opinion, and belief aboutcollected data Consider the interactive nature of threatsPHMSA RISK MODEL WORK GROUP38

Risk Assessment of a Pipeline SystemPHMSA RISK MODEL WORK GROUP39

Simplified Process WorkflowPHMSA RISK MODEL WORK GROUP40

Data Entity RelationshipsPHMSA RISK MODEL WORK GROUP41

Process is Readily AutomatedPHMSA RISK MODEL WORK GROUP42

Node by Node Data Quality Consideration Conceptually, greater data quality measure value means higher confidence about the data, which is equivalent to smaller 𝜎 value inlikelihood function Assume the data quality of each threat is determined by a Beta distribution 𝐵𝑒𝑡𝑎 𝛼𝑖 , 𝛽𝑖𝛼𝑖𝐷𝑄𝑖 𝑚𝑒𝑎𝑛 𝛼𝑖 𝛽𝑖 𝛼𝑖 and 𝛽𝑖 will be updated based on the value of data quality measures such as authenticity, compliance, reliability, transparency, andpedigree For continuous nodes, introduce a measurement error 𝜎(1 "DQ" ) For ranked nodes, adjust probability histogram to increase/decrease uncertaintyPHMSA RISK MODEL WORK GROUP43

Data Quality Effect – Can be Extended to ConsequenceMeasures for Benefit/Cost Analysis of Improved Data QualityPHMSA RISK MODEL WORK GROUP44

Direct Tie in to Six Sigma Approaches To investigate the effect of likelihood function, it is assumedthat the pressure of a pipeline system follow a normaldistribution 𝜃 𝑁 𝜇0 , 𝜎0 2 Assume the value from the measurement system 𝑥 ′ 𝑥′ 𝜃 𝜀 where the measurement error term 𝜀 𝑁 0, 𝜎 Therefore, the posterior distribution of the pressure given themeasurement 𝑥 ′ is given as𝜎 2 𝜇0 𝜎0 2 𝑥 ′ 2(𝜃 )(𝜃 𝜇0 )2(𝑥 ′ 𝜃)2𝜎0 2 𝜎 2𝑝 𝜃 𝑥 exp( )exp( ) exp( )𝜎 2𝜎 22 𝜎0 22 𝜎22 2 0 2𝜎0 𝜎 The uncertainty of the posterior distribution depends on𝜎2 𝜎0 2.𝜎0 2 𝜎 2 It is trivial to show that values of 𝜎 produce higher uncertaintyin the posterior distribution 𝜃PHMSA RISK MODEL WORK GROUP45

Subject Matter ExpertisePHMSA RISK MODEL WORK GROUP46

Biases and Heuristics (Kahneman and Tversky 1982) Ambiguity effect—The avoidance of options for which missinginformation makes the probability seem unknown. Attentional bias—Neglect of relevant data when making judgments of a correlation orassociation. Availability heuristic—Estimating what is more likely bywhat is more available in memory, which is biased toward vivid, unusual,or emotionally charged examples. Base rate neglect—Failing to takeaccount of the prior probability. This was at the heart of the commonfallacious reasoning in the Harvard medical study described in Chapter 1.It is the most common reason for people to feel that the results ofBayesian inference are nonintuitive. Bandwagon effect—Believingthings because many other people do (or believe) the same. Related togroupthink and herd behavior. Confirmation bias—Searching for orinterpreting information in a way that confirms one’s preconceptions. Déformation professionnelle—Ignoring any broader point of view andseeing the situation through the lens of one’s own professional norms. Expectation bias—The tendency for people to believe, certify, andpublish data that agrees with their expectations. This is subtly different toconfirmation bias, because it affects the way people behave before theyconduct a study. Framing—By using a too narrow approach ordescription of the situation or issue. Also framing effect, which is drawingdifferent conclusions based on how data are presented. Need forclosure—The need to reach a verdict in important matters; to have ananswer and to escape the feeling of doubt and uncertainty. The personalcontext (time or social pressure) might increase this bias. Outcomebias—Judging a decision by its eventual outcome instead of based onthe quality of the decision at the time it was made. Overconfidenceeffect—Excessive confidence in one’s own answers to questions. Forexample, for certain types of question, answers that people rate as 99%certain turn out to be wrong 40% of the time. Status quo bias—Thetendency for people to like things to stay relatively the same.Fenton, Norman. Risk Assessment and Decision Analysis with Bayesian Networks (Page 262). Taylor and Francis CRC ebook account. Kindle EditionPHMSA RISK MODEL WORK GROUP47

Comparing Expert Models It is possible to construct hybrid Bayesiannetworks that compare the performance ofcompeting expert models given different dataFenton, Norman. Risk Assessment and Decision Analysis with Bayesian Networks (Page 320 - 324).Taylor and Francis CRC ebook account. Kindle EditionPHMSA RISK MODEL WORK GROUP48

OptimizationPHMSA RISK MODEL WORK GROUP49

How to Optimize Risk Reduction Activities This is a vast topic with many well defined solution approaches that wecannot even begin to address adequately in a single session Monte Carlo simulation can be used to good effect in evaluating a largenumber of alternative approaches It is important to evaluate many diverse approaches and identify whichapproach is best in particular situations (SITUATIONAL AWARENESS) Constant Bayesian updating of the underlying probability distributionsand re-running simulations is helpful Causal networks will help define the optimization problem better Approaches that reduce system variance will have great effect Prompt mitigation and repair of identified system anomalies will helpreduce the likelihood of higher order system interactions that can beproblematicPHMSA RISK MODEL WORK GROUP50

Applying the Causal Framework to ArmageddonFenton, Norman. Risk Assessment and Decision Analysis with Bayesian Networks (Page 46). Taylor and Francis CRC ebook account. KindleEdition. Risk measurement is more meaningful in thecontext; the BN tells a story that makes sense.This is in stark contrast with the simple “riskequals probability times impact” approachwhere not one of the concepts has a clearunambiguous interpretation. Uncertainty isquantified and at any stage we can simply readoff the current probability values associated withany event. It provides a visual and formalmechanism for recording and testing subjectiveprobabilities. This is especially important for arisky event that you do not have much or anyrelevant data about (in the Armageddonexample this was, after all, mankind’s firstmission to land on a meteorite).PHMSA RISK MODEL WORK GROUP51

KUUUB Factors and Other operational RisksFenton, Norman. Risk Assessment and Decision Analysis with Bayesian Networks (Chapter 11). Taylor and Francis CRC ebook account.Kindle Edition. Known Unknowns Unknown Unknowns BiasPHMSA RISK MODEL WORK GROUP52

Discussion and QuestionsPHMSA RISK MODEL WORK GROUP53

Additional Slides On IntegratingData Quality into Risk AsessmentPHMSA RISK MODEL WORK GROUP54

Stochastic Risk Analysis ofGas Pipeline with BayesianStatisticsPHMSA RISK MODEL WORK GROUP55

Deterministic Risk Analysis Most common quantitative risk assessment method Estimates single-point value for discrete scenarios such as worst case, bestcase, and most likely outcomes Considers only a few outcomes, ignoring all other possibilities Disregards interdependence between inputs Ignores uncertainty in input variablesPHMSA RISK MODEL WORK GROUP56

Stochastic Risk Analysis with Bayesian Statistics Advanced quantitative risk assessment method Estimates probability distribution of the risk By using probability distributions, variables can have different probabilities of differentoutcomes Uncertainties in variables are described by probability distributions Probabilistic results show not only what could happen, but how likely each outcome is Allows scenario analysis and sensitivity analysis Possible to model interdependent relationships between input variablesPHMSA RISK MODEL WORK GROUP57

Bayesian Statistics Quantitative tool to rationally update subjective prior beliefs in light of new evidence.Posterior Prior Likelihood𝑃 𝐷 𝑃 𝐷 𝑃( )/𝑃(𝐷) is the parameterP( ) is the priorP( D) is the posteriorP(D ) is the likelihoodP(D) is the evidencePHMSA RISK MODEL WORK GROUP58

Illustrative Pipeline Network for Example in FollowingSlides Synthetic gas pipeline networkof 120 components (e.g.,joints, piping, etc) divided into: 3 regions 4 segments per region 10 components per segment Risk analysis can be performedat component level, segmentlevel, or region levelPHMSA RISK MODEL WORK GROUP59

Component Failure Threat Types from ASME B31.8SStandard Time-Dependent Threats Internal Corrosion External Corrosion Stress Corrosion Cracking Time-Independent Threats Incorrect Operations Procedure Weather and Outside Force Third Party Damage Resident Threats Manufacturing Defects Construction or Fabrication Defects Equipment FailurePHMSA RISK MODEL WORK GROUP60

Probability of Component Failure Probability of Failure over time 𝑡𝑝𝑛𝑚 𝑗 𝑡 𝑖 1 𝑘 1 𝑤𝑗, 𝑘1𝑓 𝑡𝑖 , 𝑥𝑗, 𝑦𝑘 𝑑𝑦 𝑑𝑥 𝑑𝑡 x, y, and t represents threat type, input variable, and time instance respectively. [t1, tp] is the time interval for which likelihood of failure is calculated n is the total number of threat type ( 9 for gas pipeline per AMSE B31.8S) m is the total number of input variables responsible for a given threat type where wj,k is the weight applied for each input variable from threat model f(t, x, y) is calculated from the Beta distribution of component failure attribute asshown in the next few slides.PHMSA RISK MODEL WORK GROUP61

Probability of Component Failure1. Determine prior belief P( ) on parameters [0,1] where is always eithersuccess (1) or failure (0) Beta distribution quantifies the prior beliefs for binomial outcome. The probability density function of the beta distribution is𝑃(𝜃 𝛼, 𝛽) 𝜃α 1(1 𝜃)β 1/𝐵(𝛼, 𝛽)where 𝐵(𝛼, 𝛽) acts a normalizing constant so that the area under PDFsums to one. Initially, ignorant prior of 𝐵(𝛼 1, 𝛽 1) is used in the very first run.PHMSA RISK MODEL WORK GROUP62

Probability of Component Failure2. Determine likelihood function. Bernoulli distribution is well-suited for the Boolean-valued outcomeusually labelled as ‘success’ (1) and ‘failure’ (0), in which it takes thevalue 1 with probability p and the value 0 with probability 1-p. The probability mass function of the distribution is𝑓 𝑘, 𝑝 𝑝𝑘 1 𝑝where 𝑘 {1,0}.PHMSA RISK MODEL WORK GROUP1-k63

Probability of Component Failure3. Determine posterior probability function of [0,1]. Posterior Prior Likelihood𝑃 𝑃 𝐷 𝑃 𝐷 𝑤ℎ𝑒𝑟𝑒, P(D) 𝑃 𝐷, 𝑑 𝑃 𝐷 Bernoulli likelihood and Beta prior are conjugate pairs – as a result, the posterior is a Betadistribution. The conjugate priors simplifies the calculation of posterior distribution. The computation of posterior distribution is a complex process (example, integration forP(D)) and therefore a numerical approximation method instead such as Markov ChainMonte Carlo (MCMC) is needed. 𝑃𝑜𝑠𝑡𝑒𝑟𝑖𝑜𝑟 𝐵𝑒𝑡𝑎 𝑑𝑖𝑠𝑡𝑟𝑖𝑏𝑢𝑡𝑖𝑜𝑛 𝑀𝐶𝑀𝐶 𝐵𝑒𝑡𝑎 𝑝𝑟𝑖𝑜𝑟, 𝐵𝑒𝑟𝑛𝑜𝑢𝑙𝑙𝑖 𝑙𝑖𝑘𝑒𝑙𝑖ℎ𝑜𝑜𝑑PHMSA RISK MODEL WORK GROUP64

Probability of Component Failure using Markov ChainMonte CarloMetropolis-Hastings algorithm1. Begin the algorithm at the current position in parameter space (θcurrent)2. Propose a "jump" to a new position in parameter space (θnew)3. Accept or reject the jump probabilistically using the prior information andavailable data4. If the jump is accepted, move to the new position and return to step 15. If the jump is rejected, stay at current position and return to step 16. After a set number of jumps have occurred, return all ofthe accepted positionsPHMSA RISK MODEL WORK GROUP65

Probability Distribution of Component FailurePHMSA RISK MODEL WORK GROUP66

Consequence of FailuresFive Categories of Consequences Very Low Low Medium High Very HighPHMSA RISK MODEL WORK GROUP67

Stochastic Risk Analysis𝑅𝑖𝑠𝑘 Probability 𝑜𝑓 𝐹𝑎𝑖𝑙𝑢𝑟𝑒 𝐶𝑜𝑛𝑠𝑒𝑞𝑢𝑒𝑛𝑐𝑒 𝑜𝑓 𝐹𝑎𝑖𝑙𝑢𝑟𝑒PHMSA RISK MODEL WORK GROUP68

Implication of Data Quality in Risk Analysis Integrity Authenticity Compliance Transparency Reliability PedigreePHMSA RISK MODEL WORK GROUP Beta Distribution 𝐵 𝛼, 𝛽 Start with 𝛼 1, 𝛽 1 𝛼 𝛼 1 𝑖𝑓 Authenticity 𝑇𝑅𝑈𝐸 1 𝑖𝑓 Compliance 𝑇𝑅𝑈𝐸 1 𝑖𝑓 Transparency 𝑇𝑅𝑈𝐸 1 𝑖𝑓 Reliability 𝑇𝑅𝑈𝐸 1 (𝑖𝑓 Pedigree 𝑇𝑅𝑈𝐸 ) 𝛽1111 𝛽 1 𝑖𝑓 Authenticity 𝐹𝐴𝐿𝑆𝐸 𝑖𝑓 Compliance 𝐹𝐴𝐿𝑆𝐸 𝑖𝑓 Transparency 𝐹𝐴𝐿𝑆𝐸 𝑖𝑓 Reliability 𝐹𝐴𝐿𝑆𝐸 𝑖𝑓 Pedigree 𝐹𝐴𝐿𝑆𝐸69

This is a Bernoulli process A Bernoulli process is a finite or infinite sequence of independent random variables X 1, X 2, X 3, ., such that For each i, the value of X i is either 0 or 1; For all values of i, the probability that X i 1 is the same number p. In other words, a Bernoulli process is a sequence of independent identically .