WIXLIB

Discussion / QuestionsSPM: SPIs-ALoSPSSP High Consequence Safety Indicator Example (with Alert and Target Setting eceding YearAllAllOperatorsOperators Mandatory IncidentTotal 210.000.1951,3537.000.1453,0069.000.17Ave 0.16SD 0.06Ave 1SD0.23Ave 2SD Ave 3SD0.290.35Current Year Alert Level setting criteria is:Preceding Year Ave 1/2/3 60.160.16Current yearAllAllOperatorsOperators MandatoryMth Total nt Year Alert LevelsPreceding Preceding PrecedingIncident Year Ave Year Ave Year AveRate* 1SD (line) 2SD (line) 3SD h Consequence Safety IndicatorsSafety Indicator (SI) Description1 CAA aggregate Air Operatorsmonthly accident/ seriousincident rate [per 1000 52 CAA aggregate Aerodromesmonthly ground accident/serious incidents rate - involvingany aircraft [per 10,000 groundmovements]3 CAA aggregate ATS monthlyFIR serious incidents rate involving any aircraft [per100,000 air movements]0.15Ave 3 SDAve 2 SDAve 1 SDTarget192009 Average Rate 1/2/3 SD. (annualreset)43 % improvement ofthe 2010 Average Rateover the 2009 AverageRate.2009 Average Rate 1/2/3 SD. (annualreset)04 % improvement ofthe 2010 Average Rateover the 2009 AverageRate.812Sub-totalMaxLower Consequence Safety IndicatorsAlert levelNotSI Alert Level/SI Target level/BreachedCriteria (for 2010)criteria (for 2010)[Yes (2),No (0)]CAA aggregate Air Operators 25% Average LEI; 10% Average LEI;Organization annualOR any level 1 finding;2AND 1 level 2surveillance/ audit outcomesOR 5 level 2 findingsfinding per auditper audit 25% Average LEI;CAA annual Air Operator LineOR any level 1 finding;Station surveillance inspection2 10% Average LEIOR 5 level 2 findingsAve LEI% (for each Operator).per audit 25% Average LEI;OR any level 1 finding;CAA annual Foreign AirNot less than 50% ofOR 5 level 2 findingsOperators Ramp sampling2Foreign Operators toper audit; OR 25% ofinspection program.be inspectedForeign OperatorsinspectedCAA aggregate Aerodrome 25% Average LEI; 10% Average LEI;Operators Organization annual OR any level 1 finding;0AND 1 level 2surveillance/ audit outcomesOR 5 level 2 findingsfinding per auditper auditCAA aggregate ATS Operators 25% Average LEI; 10% Average LEI;Organization annualOR any level 1 finding;2AND 1 level 2surveillance/ audit outcomesOR 5 level 2 findingsfinding per auditper auditCAA aggregate ATS quarterly5 % improvement of2009 Average Rate FIR TCAS RA incidents rate the 2010 Average Rate1/2/3 SD. (annual2involving any aircraft [perover the 2009 Averagereset)10,000 flight movements]Rate.CAA aggregate D&M/ MRO 25% Average LEI; 10% Average LEI;Organization annualOR any level 1 finding;2AND 1 level 2surveillance/ audit outcomesOR 5 level 2 findingsfinding per auditper auditCURRENT YEAR ALLOPERATORS MANDATORYINCIDENT RATE (PER 1000 decnovoctsepaugjuljunmayaprfebmarjandecCAA aggregate AMO (MRO)quarterly rate of componentwarranty claims due to (Major)technical defects.PRECEDING YEAR ALLOPERATORS MANDATORYINCIDENT RATE (PER 1000FH)Alert levelNotSI Target level/Breachedcriteria (for 2010)[Yes (4),No (0)]5 % improvement ofthe 2010 Average Rate4over the 2009 AverageRate.Safety Indicator (SI) Description0.400.350.300.250.200.150.100.050.00jan feb mar apr may jun jul aug sep oct nov dec2009 Average Rate 1/2/3 SD. (annualreset)Sub-totalMax* Rate Calculation:( per 1000 FH)Current Year Target is say 5% Ave rateimprovement over the Ave rate for thepreceding year, which is:SI Alert Level/Criteria (for 2010)2009 Average Rate 1/2/3 SD. (annualreset)Sub-totalMaxNo Alert %0121671.4%Overall ALoS Performance5 % improvement ofthe 2010 Average Rateover the 2009 AverageRate.Sub-totalMaxTarget Achieved %57.8%TargetAchieved[Yes (3),No (0)]03039TargetAchieved[Yes (1),No (0)]010010103835.3%

SSP Safety Performance Indicators and ALoSPContents:Sheet 1 Instructions for WGsSheet 2 SSP-SMS Safety Performance Indicators correlation Sheet 3 Safety Indicator Chart Sheet 4 SPI Data Sheet Sheet 5A SPIs Summary Package Performance (ALoSP)3-Apr-15 Purpose –Template for the selection and development of individual SPIs with harmonized Alert & Target setting procedure.Include procedure for consolidation of the SPIs’ individual Alert and Target performance outcomes to derive theALoSP (applicable for SSP & SMS).

Safety Indicators Development and ALoS Performance MonitoringInstructions for Working Groups:12345678Apr-15Browse SMM 4.3.5 and C4-App4 for detailed GM on this subject.This exercise is to develop one example of a high consequence SSP/ SMS Safety Performance Indicator chart, asin Sheet 3.Elect a group discussion coordinator and also have some one (with laptop) to make entries in the SPI datasheet (Sheet 4).Browse examples of SPIs for the various sectors, as listed in Sheet 2 attached (or other sources).Decide on one high consequence (accidents/ serious incidents data trend) SPI chart to be developed.Fill out the SPI data sheet (Sheet 4) with required details for your intended SPI chart i.e occurrence numbersand FH/ FC/ Air or Grd Movements. Actual data preferred, otherwise provide approximate viable figures basedon experience. Customize column descriptions to suit your SPI description.If time permit, develop another lower consequence SPI chart (routine incidents, etc) using the same procedure.(Browse lower consequence SPI examples column in Sheet 2)Be prepared to share your completed SPI charts and comments with others.

Examples of Safety Indicators (SSP) & Safety Performance Indicators (SMS)SSP Safety Indicators (Aggregate State)High Consequence Indicators(Occurrence/ Outcome-based)ETCAerodrome OperatorsAir Operator Individual Fleetmonthly serious incident rate(eg per 1000FH)Ave 1/2/3 SD. % (eg 5%)(annual or 2improvementyearly reset)between eachannual MeanRate.Operator Combined Fleetmonthly Incident rate (eg per1000FH)Air Operator Combined Fleetmonthly serious incident rate(eg per 1000FH)Ave 1/2/3 SD. % (eg 5%)(annual or 2improvementyearly reset)between eachannual MeanRate.Operator Internal QMS annualaudit LEI% or findings rate(findings per audit)ConsiderationConsiderationAir Operator Engine IFSDincident rate (eg per 1000 FH)Ave 1/2/3 SD. % (eg 5%)(annual or 2improvementyearly reset)between eachannual MeanRate.Operator Voluntary Hazardreports rate [eg per 1000 FH]ConsiderationConsiderationAlert level Target levelcriteriacriteriaConsiderationAlert level Target level Safety Performancecriteriacriteria IndicatorConsiderationAve 1/2/3 SD. % (eg 5%)(annual or 2improvementyearly reset)between eachannual MeanRate.CAA aggregate Air Operatorannual Line StationInspection LEI% or findingsrate (findings per inspection)Lower Consequence Indicators(Event/ Activity-based)ConsiderationCAA aggregate AirOperators quarterly IFTBincident rate (eg per 1000FH)% (eg 5%)improvementbetween eachannual MeanRate.CAA aggregate Air Operatorannual surveillance AuditLEI% or findings rate(findings per audit)ConsiderationAve 1/2/3 SD. % (eg 5%)(annual or 2improvementyearly reset)between eachannual MeanRate.Alert level Target level Safety Performancecriteriacriteria IndicatorConsiderationAlert level Target levelSafety IndicatorcriteriacriteriaCAA aggregate AirAve 1/2/3 SD.Operators quarterly Engine (annual or 2IFSD incident rate (eg per yearly reset)1000 FH)High Consequence Indicators(Occurrence/ Outcome-based)Lower Consequence Indicators(Event/ Activity-based)Air Operators (Air Operators of the State only)CAA aggregate AirOperators monthly seriousincident rate (eg per1000FH)SMS Safety Performance Indicators (Individual Service Provider)Air Operator Air Turn Back(ATB, technical) incident rate(eg per 1000 FH)Ave 1/2/3 SD. % (eg 5%)(annual or 2improvementyearly reset)between eachannual MeanRate.Operator DGR incident reports Ave 1/2/3 SD. % (eg 5%)rate [eg per 1000 FH](annual or 2improvementyearly reset)between eachannual MeanRate.CAA annual Foreign AirOperators Rampsurveillance inspection AveLEI% (for each ForeignOperator).ConsiderationSafety Indicator6Mar14CAA aggregate Operators'DGR incident reports rate[eg per 1000 FH]Ave 1/2/3 SD. % (eg 5%)(annual or 2improvementyearly reset)between eachannual MeanRate.Air Operator RTO (high speed) Ave 1/2/3 SD. % (eg 5%)incident rate (eg per 1000 FH) (annual or 2improvementyearly reset)between eachannual MeanAve 1/2/3 SD. % (eg 5%)(annual or 2improvementyearly reset)between eachannual MeanRate.

SSP Safety Indicators (Aggregate State)High Consequence Indicators(Occurrence/ Outcome-based)High Consequence Indicators(Occurrence/ Outcome-based)Lower Consequence Indicators(Event/ Activity-based)% (eg 5%)improvementbetween eachannual MeanRate.Aerodromes Operator quarterly Ave 1/2/3 SD.Runway Excursion incidents(annual or 2rate - involving any aircraft [eg yearly reset)per 10,000 departures]% (eg 5%)improvementbetween eachannual MeanRate.Aerodrome Operator quarterlyRunway Foreign Object Report(FOR) rate [eg per 10,000ground movements]ConsiderationCAA aggregate aerodromes Ave 1/2/3 SD.monthly/ quarterly Runway (annual or 2Incursion incidents rate yearly reset)involving any aircraft [eg per10,000 departures]% (eg 5%)improvementbetween eachannual MeanRate.Aerodromes Operator quarterly Ave 1/2/3 SD.Runway Incursion incidents rate (annual or 2- involving any aircraft [eg per yearly reset)10,000 departures]% (eg 5%)improvementbetween eachannual MeanRate.Operator Voluntary Hazardreports rate [per operationalpersonnel per quarter]ConsiderationCAA aggregate aerodromes Ave 1/2/3 SD.monthly/ quarterly Runway (annual or 2Excursion incidents rate yearly reset)involving any aircraft [eg per10,000 departures]ConsiderationAerodrome Operator InternalQMS annual audit LEI% orfindings rate (findings per audit)ConsiderationAve 1/2/3 SD. % (eg 5%)(annual or 2improvementyearly reset)between eachannual MeanRate.Alert level Target levelcriteriacriteria% (eg 5%)improvementbetween eachannual MeanRate.ConsiderationAerodrome Operator quarterlyground accident/ seriousincident rate - involving anyaircraft [eg per 10,000 groundmovements]Alert level Target level Safety Performancecriteriacriteria IndicatorCAA aggregate aerodromes Ave 1/2/3 SD.quarterly ground accident/ (annual or 2serious incidents rate yearly reset)involving any aircraft [eg per10,000 ground movements]ConsiderationCAA aggregate AerodromeOperators annualsurveillance Audit LEI% orfindings rate (findings peraudit)Alert level Target level Safety Performancecriteriacriteria IndicatorConsiderationAlert level Target levelSafety IndicatorcriteriacriteriaLower Consequence Indicators(Event/ Activity-based)ConsiderationSafety IndicatorSMS Safety Performance Indicators (Individual Service Provider)

SSP Safety Indicators (Aggregate State)High Consequence Indicators(Occurrence/ Outcome-based)Safety IndicatorSMS Safety Performance Indicators (Individual Service Provider)High Consequence Indicators(Occurrence/ Outcome-based)Lower Consequence Indicators(Event/ Activity-based)Alert level Target levelSafety IndicatorcriteriacriteriaAlert level Target level Safety Performancecriteriacriteria IndicatorLower Consequence Indicators(Event/ Activity-based)Alert level Target level Safety Performancecriteriacriteria IndicatorAlert level Target levelcriteriacriteriaAerodrome Operator quarterly Ave 1/2/3 SD.aircraft ground Foreign Object (annual or 2Damage (FOD) incident report yearly reset)rate - involving damage toaircraft [eg per 10,000 groundmovements]% (eg 5%)improvementbetween eachannual MeanRate.CAA aggregate ATSAve 1/2/3 SD.quarterly FIR TCAS RA(annual or 2incidents rate - involving any yearly reset)aircraft [eg per 100,000 flightmovements]% (eg 5%)improvementbetween eachannual MeanRate.ATS provider quarterly FIRAve 1/2/3 SD.serious incidents rate - involving (annual or 2any aircraft [eg per 100,000yearly reset)flight movements]Ave 1/2/3 SD. % (eg 5%)(annual or 2improvementyearly reset)between eachannual MeanRate.CAA aggregate ATSAve 1/2/3 SD.quarterly FIR Level Bust(annual or 2(LOS) incident rate yearly reset)involving any aircraft [eg per100,000 flight movements]% (eg 5%)improvementbetween eachannual MeanRate.ATS provider quarterly/ annual Ave 1/2/3 SD. % (eg 5%)near miss incident rate [eg per (annual or 2improvement100,000 flight movements]yearly reset)between eachannual MeanRate.ATS Operator quarterly FIRAve 1/2/3 SD.Level Bust (LOS) incident rate - (annual or 2involving any aircraft [eg peryearly reset)100,000 flight movements]% (eg 5%)improvementbetween eachannual MeanRate.ATS provider quarterly/ annual Ave 1/2/3 SD.GPWS/ EGPWS incident rate (annual or 2[eg per 100,000 flightyearly reset)movements]ATS Operator Internal QMSannual audit LEI% or findingsrate (findings per audit)ConsiderationCAA aggregate quarterlynear miss incident rate [egper 100,000 flightmovements]% (eg 5%)improvementbetween eachannual MeanRate.CAA aggregate ATSOperators annualsurveillance Audit LEI% orfindings rate (findings peraudit)% (eg 5%)improvementbetween eachannual MeanRate.% (eg 5%)improvementbetween eachannual MeanRate.ATS Operator quarterly FIRTCAS RA incidents rate involving any aircraft [eg per100,000 flight movements]Ave 1/2/3 SD. % (eg 5%)(annual or 2improvementyearly reset)between eachannual MeanRate.ConsiderationCAA aggregate ATSAve 1/2/3 SD.quarterly FIR (airspace)(annual or 2serious incidents rate yearly reset)involving any aircraft [eg per100,000 flight movements]ConsiderationATS OperatorsConsiderationETC

SSP Safety Indicators (Aggregate State)High Consequence Indicators(Occurrence/ Outcome-based)High Consequence Indicators(Occurrence/ Outcome-based)Lower Consequence Indicators(Event/ Activity-based)Alert level Target levelSafety IndicatorcriteriacriteriaAlert level Target level Safety Performancecriteriacriteria IndicatorLower Consequence Indicators(Event/ Activity-based)Alert level Target level Safety Performancecriteriacriteria IndicatorMRO/ POA/ DOA InternalQMS annual audit LEI%or findings rate (findingsper audit).MRO/ POA/ DOAquarterly final inspection/testing failure/ rejectionrateConsiderationConsiderationMRO/ POA/ DOAVoluntary Hazard reportsrate [per operationalpersonnel per quarter]ConsiderationConsiderationMRO/ POA quarterly rateof component Mandatory/Major Defect Reportsraised.% (eg 5%)improvementbetween eachannual MeanRate.ConsiderationConsiderationMRO/ POA quarterly rate Ave 1/2/3of component technicalSD. (annualwarranty claims.or 2 yearlyreset)ConsiderationCAA aggregate POA/DOA quarterly rate ofoperational productswhich are subject ofAirworthinessDirectives (ADs) [perproduct line]CAA aggregate MRO/POA/ DOA annualsurveillance AuditLEI% or findings rate(findings per audit)Consideration% (eg 5%)improvementbetween eachannual MeanRate.ConsiderationAve 1/2/3SD. (annualor 2 yearlyreset)ConsiderationPOA/ DOA/ MRO OrganizationsCAA aggregate MROquarterly MandatoryDefect Reports (MDR)receivedConsiderationOther examples - MinimumSeparation Infringement,Airspace infringement,Deviation from ATC clearance,Deviation from AirspaceETCETCAlert level Target levelcriteriacriteriaConsiderationSafety IndicatorSMS Safety Performance Indicators (Individual Service Provider)

SSP High Consequence Safety Indicator Example (with Alert and Target Setting Criteria)/ SafetyIndicators N ALoS Performance v90.400.350.300.250.200.150.100.050.00Ave 3 SDAve 2 SDAve 1 SDTargetjan feb mar apr may jun jul aug sep oct nov decPRECEDING YEAR ALL OPERATORS MANDATORYINCIDENT RATE (PER 1000FH)PRECEEDING YEAR AVERAGE (AVE)dec jan feb mar apr may jun jul aug sep oct nov decCURRENT YEAR ALL OPERATORS MANDATORYINCIDENT RATE (PER 1000 FH)CURRENT YEAR TARGET AVERAGEA) Alert Level Setting:C) Target Level Setting(Planned Improvement) :Alert level for a new monitoring period (current year) is based on the precedingperiod's performance (preceding year), namely its data points Average & StdDeviation. The 3 alert lines are Ave 1SD, Ave 2SD and Ave 3SDTarget setting may be less structured than Alert level setting - eg target the new(current year) monitoring period's Ave rate to be say 5% lower (better) than thepreceding period's Ave value.B) Alert Level Trigger:D) Target Achievement:An Alert (abnormal/ unacceptable trend) is indicated if ANY of the conditions beloware met for the current monitoring period (current year):- Any single point is above 3 SD line- 2 consecutive points are above 2 SD line- 3 consecutive points are above 1 SD lineAt end of the current year, if the Ave rate for the current year is at least 5% or morelower than the preceding year's Ave rate, then the set Target of 5% improvement isdeemed to have been achieved.When an Alert is triggered (potential high risk or out of control situation), appropriatefollow-up action is expected, such as further analysis to determine source and rootcause of the abnormal incident rate and any necessary action to address theunacceptable trend .Alert & Target levels should be reviewed/reset for each new monitoring period, basedon the equivalent preceding period's Ave rate & SD, as applicable.E) Alert & Target Levels - Validity Period:Contents Data Sht Summary

SSP High Consequence Safety Indicator Example (with Alert and Target Setting eceding YearAllOperatorsAllOperators Mandatory Incident AveTotal FH IncidentsRate* 0.140.1653,0069.000.170.16Ave 0.16SD 0.06Ave 1SD0.23Ave 2SD Ave 3SD0.290.35Current Year Alert Level setting criteria is:Preceding Year Ave 1/2/3 SDCurrent yearAllAllOperatorsOperators MandatoryIncidentsMth Total 17611may4746913junjulaugsepoctnovdecCurrent Year Alert LevelsCurrentPrecedingYearPreceding Year Ave PrecedingTargetYear Ave 2SDIncident Year Ave(line) 3SD (line)(line)Rate* 1SD 350.150.230.290.350.15* Rate Calculation:( per 1000 FH)Current Year Target is say 5% Ave rateimprovement over the Ave rate for thepreceding year, which is:0.15SPI ChartContents ALoSP

Example: State "X" SSP ALoS Performance Summary (say for Year 2010)High Consequence Safety IndicatorsSafety Indicator (SI) Description1 CAA aggregate Air Operatorsmonthly accident/ seriousincident rate [per 1000 FH]2 CAA aggregate Aerodromesmonthly ground accident/serious incidents rate - involvingany aircraft [per 10,000 groundmovements]3 CAA aggregate ATS monthlyFIR serious incidents rate involving any aircraft [per100,000 air movements]SI Alert Level/Criteria (for 2010)2009 Average Rate 1/2/3 SD. (annualreset)2009 Average Rate 1/2/3 SD. (annualreset)2009 Average Rate 1/2/3 SD. (annualreset)Alert levelTargetNotSI Target level/AchievedBreached criteria (for 2010)[Yes

the 2010 Average Rate over the 2009 Average Rate. 3 3 CAA aggregate ATS monthly FIR serious incidents rate - involving any aircraft [per 100,000 air movements] 2009 Average Rate 1/2/3 SD. (annual reset) 0 4 % improvement of the 2010 Average Rate over the 2009 Average Rate. 0 Sub-total 8 Sub-total 3 Max 12 Max 9 SI Alert Level/ Criteria (for 2010)