Transcription
Introduction to IndividualizedQuality Control Plan (IQCP) TheNew QC Policy from CMSJane L. Smith MS MT(ASCP) SI, DLMScientific AffairsJane.L.Smith@Alere.comOctober 2013
Disclaimers I work for Alere Alere produces testingdevices for use at POC Working with customers ismy passion!
Four key CMS regulations formoderately complex tests493.1253 Test method verificationaccuracy, precision, reportable rangeand reference ranges493.1254 Maintenance and function checks493.1255 Calibration and calibration verification493.1256 QC proceduresCLIACMS 2004 brochure on how to complete the initial “performance verification.” p.2
493.1256 – QC proceduresFor each test system, the laboratory musttest, at a minimum, two levels of externalQC materials each day it performs anonwaived test.However, the regulations now allowthe laboratory to reduce thefrequency of testing external QCmaterials (equivalent QC procedure)for certain test systems.CLIACMS: Equivalent Quality Control Procedures Brochure #4
What is the future for QC of POCT?A Risk Management approach to Quality ControlNo more onesize-fits-allformulasEvaluate theQC features ofthe deviceAnalyze otherelements ofvariability thatmust becontrolledAssess theseverity offailures in eachstep of thetesting processDevise QCtesting tomonitor andcatch saidfailuresThis represents a shift from “QualityCompliance” to true Quality Control
CLSI to the Rescue!!!EP23User Defined QCProtocols for In VitroDiagnostic DevicesBased onManufacturer’s RiskMitigation Informationand the User’sEnvironment“Portions of theEP23-A documentcapture theprinciples of ourintended policies.”EP18Risk ManagementTechniques toIdentify and ControlLaboratory ErrorSourcesCLSI. Laboratory Quality Control Based on Risk Management; Approved Guideline. CLSI document EP23-A.Wayne, PA: Clinical and Laboratory Standards Institute; 2011. S&C:13-54-CLIA 2013
Do This!!!Life-Cycle Risk Management Process
08/16/13 CMS Official MemorandumKey concepts from EP-23 will be an acceptable alternativeQC policy. The New CLIA QC policy will be entitledIndividualized Quality Control Plan (IQCP)IQCPs are a formal representation and compilation ofRisk Assessment (RA), Quality Control Plan (QCP), andQuality Assessment (QA).IQCPs must at minimum have a RA evaluation of thefollowing components: specimen, environment, reagent,test system, and testing personnelIQCP will be voluntary: Laboratories will have two choicesfor QC compliance: 1) Two levels of QC per day or, 2)IQCP. Package insert requirements must be met.Education andtransition periodbegins 01/01/2014EQC will be phased out at the end of the education andtransition period on 01/01/2017.
CMS: The “Right QC” Is IQCPIQCP applies to CMScertified, nonwaivedlaboratoriesFormalizes laboratories’risk managementdecisionsIt is optional. Default isregulation 493.1256(d)(3): 2 levelsof liquid control/dayConsiders known risksmitigated bymanufacturerIncludes existing andnew analytes/testsystems andspecialties, exceptcytology/histopathologyPermits laboratories todevelop an IQCP usingtheir existing qualitypractices/informationCMS presentation at CLSI EP23 workshop, May 2012Once effective, IQCPwill supersede thecurrent EQC policy.
What Won’t Change?Existing CLIA QC and qualitysystem concepts.No regulations will change!CMS’s outcome orientedsurvey approach.Laboratories must followmanufacturers’ instructions.Laboratory director has overallresponsibility for QCP.CMS presentation at CLSI EP23 workshop, May 2012
CAP? TJC? COLA?Accreditingorganizations (AOs) andexempt states (ES)must decide toincorporate IQCP intotheir standards. Anyrelated standardchanges must beapproved by CMS.Accredited laboratories shouldcontinue to meet their accreditingorganizations’ QC standards untilthey receive notice from theirAOs.CMS presentation at CLSI EP23 workshop, May 2012S&C:13-54-CLIA 2013
Where to Obtain Information CMS/CLIA Website:http://www.cms.hhs.gov/clia/ CMS CLIA Central Office:410.786.3531 IQCP Link:IQCP@cms.hhs.gov EP23 WorkbookCMS presentation at CLSI EP23 workshop, May 2012
Risk Management approach to QCFirstunderstand thedifferencebetween hazardand risk
ManufacturerInformationTesting PersonnelRisk AnalysisRegulatoryRequirementsS&C:13-54-CLIA 2013ScientificPublicationsClinicalRequirements
Risk Management approach to QCAsk the right questions What is needed to assure quality of test results? Does the manufacturerrecommendation for QC minimize laboratory risk to an acceptable level? What are the key conditions or potential failures that could occur in thelaboratory that pose risk of harm to the patient? What is controlled/not controlled? Are validation/verification studies sufficiently robust Are EQC features sufficient to protect patient from harm? How frequently (time and replicates) should QC be tested?G. Cooper, BioRad. 2007 AACC QC Webinar
Risk Management approach to QCWhat is needed to assure qualitysafety?How frequently (time and replicates)should this be tested?What are the key conditions orpotential failures that could pose riskof harm to the patient?Are safety features sufficient toprotect from harm?What is controlled/not controlled?Are validation/verification studiessufficiently robust?G. Cooper, BioRad. 2007 AACC QC Webinar
Risk Assessment ToolsArrival to /ED Lab marker TAT Time to treatment Brainstorming Presentation of symptoms 5 Whys Fishbone diagrams Process mappingInterventionTime to evaluation/test ordering Time from marker Intervention Prep Timereceipt to diagnosis (i.e. time fromdiagnosis to cath labnotification cisionProcessG. Cooper, BioRad. 2007 AACC QC Webinar
Testing Process Fishbone reportPost-analyticalMD orders testPhlebotomy/LabelingMD receivesresultAccessioningPre-analyticalInstrument analysisAnalytical
EP23 Workbook Key Process Steps1. Operator training and competency2. Reagent/calibrator/parts procurement andstorage3. Patient sample acceptability evaluation4. System startup5. System calibration6. Loading and testing of patient samples7. Proper device function8. Test result reviewCLSI. Laboratory Quality Control Based on Risk Management; Approved Guideline. CLSIdocument EP23-A. Wayne, PA: Clinical and Laboratory Standards Institute; 2011.
EP23 Workbook Key Process Steps1. Operator training and competency2. Reagent/calibrator/parts procurement andstorage3. Patient sample acceptability evaluation4. System startup5. System calibration6. Loading and testing of patient samples7. Proper device function8. Test result reviewCLSI. Laboratory Quality Control Based on Risk Management; Approved Guideline. CLSIdocument EP23-A. Wayne, PA: Clinical and Laboratory Standards Institute; 2011.
Develop an FMEAThink in terms of the five elements of alsEquipment(Reagents and(Hardwareconsumables):and Software):Integrity,Storage,Use,Reconstitution, Maintenance,PreparationReliability(mixing), UseG. Cooper, BioRad. 2007 AACC QC ure,Humidity, Airflow, Powersupply, Waterquality
“Use”Failure Mode and EffectsAnalysis (FMEA) FlowchartPerform RiskEvaluation toDetermine if Riskis AcceptableAssemble FMEAteamIdentify FailureModes, Effects &CausesPerform RiskEstimation ofSeverity andProbability ofOccurrenceAcceptableProceed with UseUnacceptable orAcceptablewith Risk Benefit
Unacceptable orAcceptablewith Risk BenefitRecommend, Implement &Verify Risk Control ActionsEvaluate Residual RiskAcceptableRequiresRisk BenefitUnacceptableProceed with UsePerform RiskBenefitAnalysisDiscontinue Use23
FMEA flowchartPerform RiskEvaluation toDetermine if Riskis AcceptableAssemble FMEAteamIdentify FailureModes, Effects &CausesPerform RiskEstimation ofSeverity andProbability ofOccurrenceAcceptableProceed with UseUnacceptable orAcceptablewith Risk Benefit
Variables to ConsiderIntended medicaluse of test result:HIV vs triglycerideClinical setting: Mainlab, POC,Outpatient, ER, ICU,Ambulance, Nontraditional settingTime lapse: Areresult acted onimmediately or not?Testing frequency,testing personneland turnoverCondition of ancillaryequipment:Centrifuges,refrigerators, heatbathsPower requirements/fluctuationsRadio andelectromagneticwavesAge of the G. Cooper, BioRad. 2007 AACC QC Webinar
FMEA BasicsCause on FailureModes
FMEA Steps 1 and 2 Function Proper function ofreagents Failure modes Incorrect storage Expired reagents Mechanical failure Reagent drift
tsofFailureSeverityFunction FailureModes
FMEA Step #3 Function Proper function ofreagents Failure modes Incorrect storage Expired reagents Mechanical failure Reagent drift Assess the effects andseverity of each failure Falsely elevated resultsElevations x% ? Falsely depressed resultsDepressions y% ? No results delayed results Determine the cause ofeach failure (expectoverlap) and theprobability of thatoccurrence
FP, FNExpiredreagentsFP, FNMech.failureNoresultsReagentdriftFP, FNCause s ofFailureReagentfunctionFailureModesFunction
Ranking Severity of Failureand Probability of HarmNegligible Inconvenience or temporarydiscomfortMinorFrequent Once per weekProbable Temporary injury or impairment notrequiring professional medicalintervention Once per monthSeriousOccasional Injury or impairment requiringprofessional medical intervention Once per yearCritical Permanent impairment or lifethreatening injuryCatastrophic Results in patient deathISO 14971Remote Once every few yearsImprobable Once in the life of the test system
Risk Acceptability MatrixSeverity of HarmProbabilityof harmNegligible MinorSeriousCritical KOKOKXXRemoteOKOKOKOKXImprobableOKOKOKOKOKISO 14971
Rank Severity of Risk –Consequence or HarmEffectSeverity of effectHazardous, without warning May endanger patient. Involves non-compliancewith gov’t. regulation without warning.Hazardous, with warningSame as above only with warningRanking109HighMajor injury to patient requiring emergencyinterventionMinor injury to patient; patient dissatisfied7ModerateResults acceptable; not cosmetically satisfactory6Low100% of results may have to be retested; somepatient dissatisfactionTiming/efficiency defects noticed by most users5Very HighVery LowMinorVery MinorNoneSame as above, but, defect noticed by averageuserSame as above, but, defect noticed only by thediscriminating userNo effectAdapted from Quality Support Group, Inc84321
Process Occurrence – Probabilityor frequency of failureProbability ofFailurePossible FailureRatesVery high, failure isalmost inevitable 1 in 2High, repeatedfailuresModerate, occasionalfailuresLow, relatively fewfailuresRemote, unlikelyAdapted from Quality Support Group, IncCpkRankings1 in 3 0.33 0.331091 in 81 in 20 0.51 0.6787 0.83 1.00 1.17 1.33 1.50 1.676543211 in 801 in 4001 in 20001 in 15,0001 in 150,000 1 in 1,500,000
Process Detection – Probability that will bedetected before harmQualitative probabilityRemote likelihood that erroneous resultswould be undetected detection reliability at least 99.99% detection reliability at least 99.80%Low likelihood that erroneous resultswould be undetected detection reliability at least 99.5% detection reliability at least 99%Moderate likelihood of detection detection reliability at least 98% detection reliability at least 95% detection reliability at least 90%High likelihood that that erroneousresults would be undetected detection reliability at least 85% detection reliability at least 80%Extreme likelihood that erroneousresults would be undetectedAdapted from Quality Support Group, IncQuantitative probability Rankingof not 001/1005671/501/20891/10 10
Full Blown FMEA FormProcessstepfunctionPotentialfailuremodeQuality Support Group, IncPotentialcauses offailurePotentialeffects TakenOccurrenceSeverityDetectionProcessstepand/or #OccurrenceSeverityDetectionExisting conditionsRPNResponsibilityfor actiontaken
Now .What Needs Fixing? Identify those conditions that lead to unacceptablelevels of error severity and frequency. Determine operating processes or tests (quality control)to detect those conditions1st Eliminate causes of failure so that it does not OCCUR2nd Reduce probability of OCCURRENCE3rd Reduce SEVERITY of the failure4th Improve DETECTION of the failureQuality Support Group, Inc
As a Result You WillIdentify eachcomponent and itsfunctionIdentify the potentialcauses of the failure andrate the occurrence ofeachRank RPNs fromhighest to lowestG. Cooper, BioRad. 2007 AACC QC WebinarIdentify the potentialfailure mode for eachcomponentIdentify the currentcontrol for eachoccurrence and rate thedetection capabilityIdentify the potentialeffect(s) of the failureand rate the severity ofeachCalculate the RPN foreach itemTake action on thoseitems designated ashigh risk
Do This!!!Life-Cycle Risk Management Process
Other Resources ISO (www.iso.org) ISO 9000:2005 Quality Management systemsFundamentals and vocabulary ISO 14971:2007 Medical Devices-Applicationof risk management to medical devices But how are you supposed to understandall the instrument features that couldmitigate risk?J Westard, Westard QC, Inc and G. Cooper, BioRad. 2007 AACC QC Webinar
What’s Next?Device manufacturers need to provide LOTS moreinformation about their QC featuresDetailed descriptions of device risk mitigation featuresIdentify the targeted failure mode for each mitigationDescriptions of how the risk mitigation feature or recommended actionperforms its intended functionKnown limitations of the risk mitigation feature or recommended actionStudies performed to verify the feature or recommended action achieves theintended purposeG. Cooper, BioRad. 2007 AACC QC Webinar
EP22-Presentation ofManufacturer's Risk MitigationInformation for Users of in vitroDiagnostic Devices Guidance to Vendors Document design features that detectand/or control test system variabilityand/or failures. Describe failure modes, risk reductionfeatures and data to support theeffectiveness of those features.
EP22 Items Vendors WouldHave AddressedReagentdeterioration During shipment Over timeOperatorcertificationsExpiredreagentsQC sampledegredationCalibratordegredationSample dataentry sSamplecarryoverWear & tearonreplaceablepartsEnvironmental limitationsSamplelimitationsQCmaintenance
Suggested EntriesTargeted failure modeTest system feature or recommended actionDescription how the feature or recommended action isintended to functionKnown limitations of feature or recommended actionActions required to address known limitationsStudies performed to demonstrate the ability of thefeature/recommendation to achieve intended purposeSummary of study including a RA for each location wheretesting is performed on same test systems.
But What Will IQCPs Really LookLike?
Default is regulation - 493.1256(d)(3): 2 levels of liquid control/day Includes existing and new analytes/test systems and specialties, except cytology/histopathology Permits laboratories to develop an IQCP using their existing quality practices/information Considers known risks mitigated by manufacturer Formalizes laboratories' risk management .
