Transcription
CMS MMS Blueprint Measure LifecycleQuickStart
CMS MMS Blueprint Measure LifecycleQuickStartTable of ContentsTable of ContentsPageTable of Contents. iIntroduction . 1Measure Conceptualization . 4Measure Specification. 7Measure Testing. 12Measure Implementation . 16Measure Use, Continuing Evaluation, and Maintenance . 21Stakeholder Engagement. 23Conclusion. 27May 2022Page i
CMS MMS Blueprint Measure LifecycleQuickStartIntroductionIntroductionABOUT THE QUICKSTARTDesigned with both experienced and novice measure developers in mind, the CMS MMS BlueprintMeasure Lifecycle QuickStart Guide (QuickStart) provides a start-to-finish overview of qualitymeasure development, implementation, and maintenance steps and processes. Each sectionincludes information about important steps associated with a given stage of the MeasureLifecycle, along with links to additional resources, templates, and references to specific chaptersand supplemental materials of the Blueprint.The QuickStart provides tables, checklists, and procedural graphics as a tool to guide measuredevelopers through the mechanics of measure development, implementation, and maintenance.For a more comprehensive overview, visit the Blueprint content on the MMS Hub . To take adeeper dive into specific topics, view the Blueprint supplemental materials .CMS PRIORITIES & MEANINGFUL MEASURESIn nearly every setting of care, CMS is moving from paying for services to paying for value, notvolume. CMS’s goal is to foster value by promoting the highest quality, safety, and care experiencewith the most affordable, cost-efficient service possible for Americans. To do this, CMS developsquality measures (measures) that address healthcare priorities and goals and align with patientneeds. Each measure focuses on a different aspect of health care, such as processes, patient healthoutcomes, patient perceptions, and organizational structure and/or systems.The purpose of CMS measures is two-fold. First, measures promote quality and reduce waste inhealthcare by incentivizing good performance and disincentivizing poor performance through publicreporting and pay-for-reporting programs, and by allowing CMS and participating measuredentities to track performance over time. Secondly, they improve patient decision-making byproviding data through public reporting (e.g., Star Rating) to help patients, families, and caregiversmake informed decisions about where to seek care that is not just based on cost. Given this criticalrole, measures must be meaningful, robust,1 valid, feasible, based in scientific evidence, and welltested to ensure the measures do not lead to unintended negative consequences or burden forpatients or measured entities.CMS launched the comprehensive Meaningful Measures Initiative in 2017, which identifies highpriority areas for quality measurement and improvement. The purpose of this initiative was toimprove outcomes for patients, their families, and measured entities while also reducing burdenand moving payment toward value through focusing everyone’s efforts on the same quality areas.The Meaningful Measures Initiative also helped to identify and close important gap areas ofmeasures, align measures across the continuum of care and across payors, and spur innovation innew types of measures such as patient-reported measures and digital measures.1Throughout the document, “robust” refers to measures with the most vigorous quality action or guidance or as a descriptor to describe strong,vigorous, or thoroughly vetted components of a measure.May 2022Page 1
CMS MMS Blueprint Measure LifecycleQuickStartIntroductionMEASURE LIFECYCLEThe Measure Lifecycle ensures measure developers create precisely specified, valid, reliable, andsignificant measures that directly link to CMS quality goals. The Measure Lifecycle graphic, Figure 1,provides a high-level view of the major tasks involved in developing measures from the time of theinitial concept through measure implementation and maintenance . While the stages follow ageneral sequence, the process is highly iterative and allows developers the flexibility to carry outstages concurrently. Additionally, measure developers perform cross-cutting activities such asinformation gathering, stakeholder engagement, and feasibility evaluation throughout theMeasure Lifecycle.Figure 1. Key activities carried out by measure developers during the Measure Lifecycle.May 2022Page 2
CMS MMS Blueprint Measure LifecycleQuickStartIntroductionEVALUATION CRITERIAMeasure evaluation is not a single step in the Measure Lifecycle. Rather, measure developersshould apply evaluation criteria throughout the development, implementation, and maintenanceprocesses to identify weaknesses in the justification and provide an opportunity to revise andstrengthen the measure. The more effectively the measure properties meet evaluation criteria,the more likely the measure will be robust and meaningful.The evaluation criteria are Importance to measure and report: evidence and performance gaps Scientific acceptability of measure properties: reliability and validity Feasibility Usability and use Comparison to related or competing measures These criteria align with the CMS consensus-based entity (CBE) evaluation criteria. Although CMSCBE endorsement is not a requirement for use of a measure in a CMS program, CMS encouragesseeking endorsement given that endorsement indicates a level of rigor in testing and evidencethat CMS is seeking for its measures.Further, CMS expects development and testing of measures that do not seek CMS CBEendorsement to be in accordance with these evaluation criteria.For more information on the evaluation criteria, see the Measure Evaluation Criteria andGuidance for Evaluating Measures for Endorsement .May 2022Page 3
CMS MMS Blueprint Measure LifecycleQuickStartMeasure ConceptualizationMeasure ConceptualizationThe key components of measure conceptualization are information gathering and business case development.INFORMATION GATHERINGDESCRIPTIONInformation gathering includes an environmental scan (e.g., review of literature,search for clinical practice guidelines and existing measures, input from experts,and other related activities) and empirical data analysis. These activities yieldinformation that will guide the prioritization of topics or conditions, gap analysis,business case development, and compilation of related and competing measures.PURPOSEInformation gathering demonstrates the existence of a performance ormeasurement gap related to the topic of interest; it helps demonstrate measureimportance and explore feasibility. This process should yield quality goals, strengthof scientific evidence pertinent to the topics/conditions, and information withwhich to build a business case. It will also produce evidence of general agreementor conflicting views on the quality issues surrounding the topics/conditions.INFORMATION GATHERING CHECKLIST identify the healthcare quality issue and determine its priority area conduct an environmental scan analyze empirical data, as appropriate evaluate information collected during the environmental scan and empirical data analysis conduct a measurement gap analysis to identify areas for new measure development justify the creation of new measures apply measure evaluation criteria prepare an initial list of measures or measure topicsHOW TO PERFORM AN ENVIRONMENTAL SCAN: Develop a series of unambiguous, structured questions to limit the search to a specific problem set.Determine the framework for relevant work, including literature databases and search engines,keywords and phrases, inclusion and exclusion criteria, and domain experts.May 2022Page 4
CMS MMS Blueprint Measure Lifecycle QuickStartAssess the literature using qualitative techniques and quantitativemetrics such as impact (e.g., number of times a paper is cited,number of page views), innovativeness, consistency with otherworks on the topic, recency of citations used in the work,seminality/originality, and quality of writing.Qualitatively evaluate and summarize the evidence. Evaluate theeffectiveness and value of the data sources , sample sizes, datacollection methods, statistical methods, periods, and researchfindings.Interpret findings by evaluating the similarities and differencesamong the findings; then, draw conclusions to inform data collectionand analyses.Refine research questions and develop hypotheses. Generate ageneral analysis plan, including data sources and estimationprocedures.Measure ConceptualizationEXAMPLES OFINFORMATION SOURCES CMS MeasuresInventory Tool (CMIT) Environmental ScanSupport Tool (ESST) De Novo MeasureScan (DNMS) Quality PositioningSystemBUSINESS CASE DEVELOPMENTDESCRIPTIONThe business case documents the information needed to assess the anticipatedbenefits of a measure against the resources and costs required to develop andimplement a measure (burdens vs benefits).PURPOSEFacilitates decision-making about whether to invest resources in the developmentand implementation of a potential measure; helps demonstrate usability,importance, and feasibility.The measure developer initiates a business case early in measure conceptualization, updates andenhances it throughout the measure lifecycle, and uses it to compare actual results during measurereevaluation and maintenance. The business case should demonstrate why the measure is necessary the measure’s value and how it balances benefits, costs, and risks, including burden viability of the measure relative to the healthcare sector’s ability to respond whether the measure is sensitive to changes in behavior or policy such that improvements inmeasure performance reflect improvements in care delivery whether the costs of implementation are realistic whether the healthcare system has sufficient capacity to implement the measureKey elements of the business case include net benefit summary precise statement of need measure impact influencing factors resources required for measure implementation costs of clinical careTable 1 outlines several research questions measure developers should ask when developing a businesscase and key areas for which the measure could provide benefits and decrease or increase costs.May 2022Page 5
CMS MMS Blueprint Measure LifecycleQuickStartMeasure ConceptualizationTable 1. Research questions to pose during business case development.AREAS OF POTENTIAL COSTS,BENEFITS, AND SAVINGSPRIMARY RESEARCH QUESTIONSHow will this measure improve healthcare quality,processes of care delivery, outcomes of care, and/ordecrease complications or untoward effects of care?How will the measure decrease variations of care acrossdisadvantaged subgroups?How will implementing this measure decrease the cost ofcare or improve clinical efficiency?How does data collection for this measure affect clinicalworkflows?What are the far-reaching, long-term benefits of thismeasure? Does it address a gap in care not addressed inthe literature?SECONDARY RESEARCH QUESTIONSDoes the measure promote coordinated care acrosssettings?Does the measure include the patient as a member of thecare team? Patients: health outcomes,length of stay, readmissions,patient satisfaction, adverseevents, medical errors, trust ofthe healthcare system Employee and organizational:workplace safety, staff time, staffturnover, sick time, training time,turnover hiring costs, staffsupervision costs Liability: worker’s compensationclaims, liability insurancepremiums, litigation andjudgment costs, fines Materials: Product purchase,new technology or protocol,maintenance, storage, anddisposal.The cost savings analysis model is the method most commonly used to evaluate the business case.Regardless of the evaluation model, the business case should include a hypothesis that, at a minimum,states the measure’s effect over time. These details enable the measure developer to make cost-benefitdeterminations during measure use, continuing evaluation, and maintenance.REFERENCESMMS Hub Measure ConceptualizationSUPPLEMENTAL MATERIALS Environmental Scans for Quality MeasurementTEMPLATES Information Gathering Report Template Business Case Form and InstructionsMay 2022Page 6
CMS MMS Blueprint Measure LifecycleQuickStartMeasure SpecificationMeasure SpecificationThis stage of the Measure Lifecycle consists of developing the technical aspects of the measurespecifications and harmonization with related measures. Measure developers revisit the measurespecification process throughout the Measure Lifecycle to incorporate shifting measure concepts andspecifications based on testing results and changes to standards.STAKEHOLDER ENGAGEMENTIn addition to the TEP, the measure developer should engage stakeholders such as patients, caregivers,and clinicians at this stage to address the feasibility of data collection. The measure developer shouldalso consider soliciting public comments. Measure developers may post draft specifications forelectronic clinical quality measures (eCQMs) to the ONC Project Tracking SystemMEASURE SPECIFICATIONSDESCRIPTIONMeasure specifications are essentially the measure details; they include all theinformation required to define and calculate the measure. Development ofspecifications is an iterative process with testing .PURPOSEEnsures measure details are clear and unambiguous, creating a unique measurethat is distinguishable from others and to support consistent implementation.MEASURE SPECIFICATIONS CHECKLIST define the data source develop specifications and definitions specify the codes and code systems construct the data protocol document the measuresDEFINE THE DATA SOURCE: when identifying the source of data, the measure developer must considerthe feasibility and method(s) of collecting data from that source. Types of sources include administrativedata, claims data, paper patient medical records , electronic patient medical records, electronic clinicaldata, registries, and standardized patient assessments. Each source has its own benefits and limitations,such as time commitments and staff resources.DEVELOP SPECIFICATIONS AND DEFINITIONS: the construction of measure specifications begins withthe outline of the target/initial population , numerator , denominator , numerator anddenominator exclusions , denominator exceptions , and measure logic . Then, the measuredeveloper gives the measure concept increasing amounts of detail, including precisely defined dataMay 2022Page 7
CMS MMS Blueprint Measure LifecycleQuickStartMeasure Specificationelements and the appropriate values, value sets , or direct reference codes . Every part ofmeasure specification requires explicitly defined elements with accompanying analysis to identifyconstraints and criteria of the specification.Table 2. Measure specification PopulationRefers to the cohort for selecting thedenominator population. Somemeasures (e.g., ratio measures) willrequire multiple target/initialpopulations, one for the numerator andone for the denominator.All patients aged 18 years and older witha diagnosis of major depressive disorder(MDD) (CMIT Family ID 30 ) (CMS CBE0104e).DenominatorDescribes the population evaluated bythe individual measure. The populationdefined by the denominator can be thesame as the target/initial population orit can be a subset of the target/initialpopulation to further constrain thepopulation for the measure.All patients at least 18 years old as of thefirst day of the reporting month who aredetermined to be maintenancehemodialysis patients (in-center andhome HD [hemodialysis]) for thecomplete reporting month at the samefacility (CMIT Family ID 313 ) (CMS CBE2978).Format: patients, age [age or age range],with [condition] in [setting] during [timeframe]DenominatorExclusionDenominator exclusions refer to criteriathat result in removal of patients orcases from the denominator beforecalculating the numerator. An exclusionmeans that the numerator event is notapplicable to those covered by theexclusion.Patients with an active diagnosis fordepression or a diagnosis of bipolardisorder (CMIT Family ID 672 ).Format: denominator-eligible patientswho [have some additionalcharacteristic, condition, procedure]NumeratorDescribes the process, condition, event,or outcome that satisfies the measurefocus or intent.Format: patients who received/had[measure focus] {during [time frame] ifdifferent than for target population}May 2022The number of adult patient-months inthe denominator who were onmaintenance hemodialysis using acatheter continuously for three monthsor longer as of the last hemodialysissession of the reporting month (CMITFamily ID 313 ) (CMS CBE 2978).Page 8
CMS MMS Blueprint Measure SCRIPTIONAn exception permits the exercise ofclinical judgment and implies themeasured entity considered or offeredtreatment to each potentially eligiblepatient in the denominator. Exceptionsare most appropriate whenmeasurements of contraindications todrugs or procedures are relative. Onlyuse a denominator exception inproportion measures. It is notappropriate for ratio or continuousvariable measures.Measure SpecificationEXAMPLEDocumentation of medical reason(s) fornot prescribing beta-blocker therapy(e.g., low blood pressure, fluid overload,asthma, patients recently treated with anintravenous positive inotropic agent,allergy, intolerance, other medicalreasons).Documentation of patient reason(s) fornot prescribing beta-blocker therapy(e.g., patient declined, other patientreasons).Documentation of system reason(s) fornot prescribing beta-blocker therapy(e.g., other reasons attributable to thehealthcare system) (CMIT Family ID 306) (CMS CBE 0083e).NumeratorExclusionUsed only in ratio and proportionmeasures to define elements thatshould not be in the numerator data.If the number of central line bloodstreaminfections per 1,000 catheter days wereto exclude infections with a specificbacterium, that bacterium would belisted as a numerator exclusion.StratificationSchemeMeasure developers may define astratification scheme in lieu of riskadjustment by stratifying thepopulation based on their risk for anoutcome of a procedure.Measure is to be stratified by apopulation type (i.e., race, ethnicity, age,social risk factors, income, region,gender, primary language, disability)(e.g., Chlamydia Screening for Women[CMIT Family ID 128 ] [CMS CBE 0033]).SPECIFY THE CODE AND/OR CODE SYSTEMS: most measures rely at least in part on the use of variousstandardized codes or code systems for classifying healthcare provided in the United States. Themeasure developer should list all codes (plus the code system and the version that the codes camefrom) required for the measure and explicitly state the source of the codes and instructions pertainingto their use. Find more information in the Codes, Code Systems, and Value Sets supplementalmaterial.CONSTRUCT DATA PROTOCOL: the measure developer must explicitly identify the types of data andhow to aggregate or link these data so that the measure calculation can be reliable and valid. Themeasure developer should proceed carefully when merging data from different sources or systems toprevent errors in assumptions.May 2022Page 9
CMS MMS Blueprint Measure LifecycleQuickStartMeasure SpecificationDATA PROTOCOL CHECKLIST define key terms, data elements, codes, code systems describe the level of measurement/analysis describe sampling determine risk adjustment, if appropriate clearly define time intervals describe how the measure results are scored and reported develop the calculation algorithm RISK ADJUSTMENTSome measures, specifically outcome measures , may require risk adjustment. The purpose is afairer and more accurate comparison of outcomes of care across measured entities. Find moreinformation about risk adjustment modeling in the Risk Adjustment in Quality Measurementsupplemental material.DOCUMENT THE MEASURE: the measure developer must complete the detailed technical specifications,including any additional documents required to evaluate and implement the measure as intended. TheMeasure Information Form, Measure Justification Form, and several documents on the CMS CBESubmitting Standards page are available to assist in documentation of on is the standardization of specifications for related measures.Measure harmonization may be based on shared focus, target population, ordefinitions applicable to many measures so that they are uniform or compatible(unless there is a compelling reason not to, i.e., dictated by the evidence).PURPOSEHarmonization helps to reduce burden associated with measure implementationand reporting at healthcare organizations. Harmonization efforts duringdevelopment and maintenance help fulfill the CMS CBE criterion ofalignment with competing or existing measures. Find more information in theMeasure Harmonization, Respecification, and Adoption supplemental material.May 2022Page 10
CMS MMS Blueprint Measure LifecycleQuickStartMeasure SpecificationTable 3. Harmonization during measure development. N represents numerator and D represents denominator in thetable.HARMONIZATIONISSUENMEASUREsame measure focusDsame target populationNsame measure focusDdifferent targetpopulationsNdifferent measurefocussame target populationDND competingmeasure related measuredifferent measurefocusdifferent targetpopulation related measure ACTIONuse existing measure (adopted) or justifydevelopment of a new measurea different data source will require newharmonized specifications (e.g.,respecified )harmonize on measure focus (i.e.,respecified)justify differencesrespecify existing measure by expanding thetarget populationharmonize on target populationjustify differences proceed with new measure developmentnew measureREFERENCESMMS Hub Measure SpecificationSUPPLEMENTALMATERIALS Codes, Code Systems, and Value SetsComposite Measures for Accountability ProgramsCost and Resource Use MeasuresElectronic Clinical Quality Measures (eCQM) Specifications, Standards,and ToolsMeasure Harmonization, Respecification, and AdoptionMultiple Chronic Condition MeasuresPatient-Reported Outcome MeasuresRisk Adjustment in Quality Measurement May 2022Page 11
CMS MMS Blueprint Measure LifecycleQuickStartMeasure TestingMeasure TestingKey components of measure testing include developing a testing plan, alpha and beta testing,and measure evaluation.DESCRIPTIONTesting refers to all the data collection and analysis activities that contribute to theevaluation of the measure specifications.PURPOSEEnables measure developers to assess the suitability of the technical specificationsand acquire empirical evidence to help assess the strengths and challenges of ameasure with respect to the evaluation criteria, especially scientific acceptability(reliability and validity) and feasibility. Testing also provides an opportunity to buildupon earlier judgments about the measure’s importance and usability.STEPS TO PERFORM TESTING develop a plan for how to test the measure (ensure that planned methods will addressevaluation criteria) implement the testing plan analyze the test results refine the measure; incorporate stakeholder inputs retest the refined measure document adherence to measure evaluation criteria: prepare for CBE endorsement process (if applicable) compile information to support measure selection (see “Measure Implementation”)ALPHA AND BETA TESTINGTable 4 provides the features of both alpha and beta testing; measure developers should consider bothwhen developing a testing plan.Table 4. Features of alpha and beta testing.TimingScaleMay 2022ALPHA TESTING conducted prior to finalizationof technical specifications can conduct multiple times inquick succession requires only enough records toensure the presence of allelements needed for themeasure and identify commonBETA TESTING conducted after development of themeasure developer’s detailed andprecise technical specifications samples should be representative andof adequate size may require data from multiplesites/settings, depending on the dataPage 12
CMS MMS Blueprint Measure ScientificAcceptabilityFeasibilityUsability and UseMay 2022QuickStartALPHA TESTINGoccurrences or variation in thedata convenience sampling permits early detection ofproblems in technicalspecifications (e.g.,identification of additionalinclusion and exclusion criteria) may help assess volume,frequency, or costs related to ameasure topic (e.g., cost oftreating the condition, costsrelated to proceduresmeasured) establishes, on a preliminarybasis, that the measure canidentify gaps in care provides support for furtherdevelopment of the measure limited in scope if conductedduring the formative stage may include preliminaryassessment of face validityMeasure TestingBETA TESTINGsource (e.g., administrative, medicalrecord) sufficient to allow adequate testing ofthe measure’s scientific acceptability used to assess or revise thecomplexity of computations requiredto calculate the measure includes empirical evaluation ofperformance thresholds, disparities analysis, and outcome variation evaluates opportunities forimprovement in the population(e.g., by identifying variability amongcomparison groups, showing that themeasure is not “topped-out ”) empirically assesses measurereliability and validity, includinganalysis of exclusion criteria (if anyused) evaluates the risk adjustment model provides initial information provides enhanced informationregarding feasibility, including greaterabout the feasibility ofdetermination of barriers andcollecting required data andmeasured entity burden tocalculating measures usingimplementation and costs associatedtechnical specificationswith measurement identifies barriers to evaluates feasibility of stratificationimplementationfactors based on occurrences of offers an initial estimate of coststarget events in the sampleor burden of data collection andanalysis no formal analytic testing at identifies unintended consequences,this stage; may use qualitativeincluding susceptibility totesting with patients andinaccuracies and errorsmeasured entities during alpha reports strategies to amelioratetestingunintended consequences TEP may assess potential may consist of focus groups or similarusability of the measuremeans of assessing usefulness of themeasure by consumers can use the TEP to assess potentialusabilityPage 13
CMS MMS Blueprint Measure LifecycleQuickStartMeasure TestingACTIONS FOR TESTINGDEVELOP THE TESTING PLAN: a testing plan (also referred to as a work plan for testing) should includeenough information to explain how the proposed testing methodology will help meet the evaluationcriteria. Testing plans for alpha testing may look a bit different than testing plans for beta testing; at aminimum, however, all testing plans should contain the elements listed.TESTING PLAN ELEMENTS name(s) of measure(s) type of testing study objective(s) timeline for testing and report completion data collection methodology description of test population (e.g., number of test sites/data sets) description of data elements to be collected sampling methods, if applicable description of strategy to recruit measured entities/test data sets planned analysis methods and description of test statistics description and forms documenting patient confidentiality and description of InstitutionalReview Board (IRB) compliance approval and/or steps to obtain data use agreements, ifnecessary methods to comply with the Paperwork Reduction Act (PRA), if relevantANALYZE THE TEST RESULTS: when the measure developer completes data gathering from the test sites,the measure developer conducts a series of analyses to characterize the evaluation criteria of themeasures. The measure developer presents findings of testing analyses in a final summary report.REFINE THE MEASURE: the measure developer may need to modify the measure specifications, datacollection instructions, and calculation of measure results based on analysis of testing results. Forexample, following alpha testing, the measure developer may undertake measure respecification or efforts toovercome potential implementation barriers. following beta testing, changes in the definition of the population or adjustments to the comparisongroup definition may occur.May 2022Page 14
CMS MMS Blueprint Measure LifecycleQuickStartMeasure TestingSTAKEHOLDER ENGAGEMENT OPPORTUNITYIf making changes to the measure, consult with the TEP prior to retesting the measure. ViewFigure 3. Stakeholder engagement and development activities accomplished during the MeasureLifecycle. graphic for more information.RETEST THE MEASURE: measu
Measure Lifecycle QuickStart Guide (QuickStart) provides a start-to-finish overview of quality measure development, implementation, and maintenance steps and processes. . Assess the literature using qualitative techniques and quantitative metrics such as impact (e.g., number of times a paper is cited, . Research questions to pose during .
