WIXLIB

Table 1. Health Portfolio Activities, 2017 – 2022Thematic AreaMonitoring,Evaluation, &LearningHealth Portfolio ActivityCollaborating, Learning, and Adapting for Improved Health (CLAimHealth)EstimatedTimeline2018–20221.2. THEORY OF CHANGE AND RESULTS FRAMEWORKThe Theory of Change (ToC) of the Health Portfolio 2017-2022 asserts that health of Filipinos overall, includingthe health of the underserved Filipinos, will improve if: Healthy behaviors of Filipinos are strengthenedQuality of service delivery is fortifiedKey health systems are bolstered and institutionalizedThe Health Portfolio’s strategy to bring about this change is to institutionalize and strengthen the Philippinehealth system to enable healthy behaviors, quality health services, and agile health system functions. Consistentwith the Country Development Cooperation Strategy (CDCS), Partnership for Growth (PFG), and based on the2016 Health Portfolio Evaluation and the Project Appraisal Document (PAD), the strategy treats GPH as apartner who makes evidence-based decisions to bolster health systems. Additionally, the strategy involvesnational and local actors in the context of a decentralized governance system. This is a shift from the previousstrategy that provided technical assistance to GPH primarily to fill in service and management gaps.Assumptions for positive change through the Health Portfolio and its activities are as follows: Reasonable access to the underserved FilipinosContinued political support to effectively implement the Responsible Parenthood and ReproductiveHealth (RPRH) Law (Republic Act 10354), the Comprehensive TB Elimination Plan Act of 2016 (TB Law,Republic Act No. 10767), and the United Nations High-Level Meeting country targets for TBPublic sector funding at the central and regional levels remain at current levels and/or increasePublic sector services continue or resume promptly during periods of natural disaster or political unrestUSAID funding and staffing will remain at current levels and/or increaseGlobal Fund will continue to provide funding at current levels and/or increase until the public sector canadequately fill the funding gapExpected change from the implementation of the Health Portfolio Activities is measured through a set ofindicators at outcome- and output-levels for the three sub-purposes (see Figure 2 and Table 2).USAID.GOVHEALTH PORTFOLIO DATA QUALITY ASSESSMENT PLAN AND TOOLS 3

Figure 2. USAID Health Portfolio Results Framework, 2016-2022 (with performance indicators as of 03/28/2019)Health and productivity improved (CDCS IR 1.3) Total fertility rateTB incidence rateProportion of TB-affected families with catastrophic TB-related expenseImproved health for underserved Filipinos Couple-Years Protection in USG-supported programsNumber of MDR-TB cases detectedAdolescent pregnancy rate in USG-assisted sites (among women below age 18 years and 18-24 years) Healthy Behaviors Strengthened1.1 Improved individual adoption of healthybehavior Treatment success rate for MDR/RR-TBcases Percent of audience who recall hearing orseeing a specific USG-supported FP/RHmessage in USG-assisted sitesKey Health Systems Bolstered andInstitutionalizedQuality of Service Delivery Fortified2.1 Improved quality health services through patientcentered approaches Number of new FP acceptors in USGassisted sitesPercentage of new and relapse TB patients testedusing a WHO-recommended rapid test at the time ofthe diagnosis in USG-assisted sites3.1 Effective public sector supply chain formedicines, technologies and commodities rolled out Average stockout rate of contraceptive commodities atUSG-assisted FP service delivery points in USGassisted sitesNumber of MDR-TB cases that have initiated secondline treatmentPercentage of USG-assisted service delivery sitesproviding FP counseling and/or services in USGassisted sites Average stockout rate of selected anti-TB drugs and TBlaboratory tracer commodities in USG sites3.2 Critical state-of-the-art training institutionalizedand sustained by DOH Presence of Mission support to strengthen humanresources for health1.2 Improved community ownership/participation in health behaviors Number of USG-assisted CHWsproviding FP information, referrals, and/orservices during the year in USG-assistedsites Percentage of community contribution to Percent of health workers who received in-servicetraining (IST) using non-traditional learning platforms forcontinuous professional development (CPD) in familyplanning and tuberculosis in a given year in USGassisted sites2.2 Innovative approaches to improving quality ofcare tested and rolled out Average satisfaction score of TB patients currentlyundergoing treatment in USG-assisted sites Average satisfaction score of clients who came forFP/ RH services in USG-assisted sites3.3 Improved fiscal, financial, and human resourcemanagement for health institutionalized at the central,regional and local levelsTB notification Number of UHC areas supported by USG investment Proportion of PHIC claims reimbursed to total claimsfiled at USG-assisted facilities in USG-supported sitesCross-cutting indicators (see Table 2)USAID.GOVHEALTH PORTFOLIO DATA QUALITY ASSESSMENT PLAN AND TOOLS 4

Table 2. USAID Health Portfolio Cross-Cutting ity andSelf-relianceIndicatorActivity Number of joint success stories produced**Number of synergized approaches for supply chain management, human resources, for health,engagement with local government units and health financial risk protection**All Percentage (and absolute peso amounts) of local investments for health to total health investmentsin USG-assisted sites.Number of USG-assisted LGUs (province, municipalities) that are adopting and implementingactivities on sustainability/self-reliance*All Percent of men who support the use of modern contraception for themselves or for their partnersin USG-assisted sitesTB case notification rate all forms (by sex) in USG-assisted sitesPercent of gender-friendly USG-assisted service delivery points (TB and FP)*Number of USG-assisted LGUs (province, municipalities) that are adopting and implementingactivities on Gender*FPGender Supply-chainManagement Percentage of DOH ROs and PHOs implementing National SCM systems* Percentage of USG-assisted DOH-ROs and LGUs with functional ICV committees*Number of USG-assisted LGUs (province, municipalities) that are adopting and implementingactivities on ICV*FPPrivate sectorengagement Number of USG-assisted LGUs (province, municipalities) that are adopting and implementingactivities on private sector engagement*HSSClimate riskmanagement Number of USG-assisted LGUs (province, municipalities) that are adopting and implementingactivities on climate risk management*HSSInformed Choice andVoluntarism (ICV)Legend: *Proposed cross-cutting indicator as of February 2019USAID.GOVTBTB & FPHSSHSS**New coordination indicator as of December 2018HEALTH PORTFOLIO DATA QUALITY ASSESSMENT PLAN AND TOOLS 5

1.3. PURPOSE OF THE DQA PLANThe DQA Plan aims to:(1) Provide standard definitions of terms related to data quality(2) Describe standard approach and tools for A/CORs to conduct DQAs(3) Describe a minimum set of data quality processes and propose a DQA self-assessment tool forimplementing partners (IPs)(4) Propose priority indicators for the Health Portfolio DQA(5) Describe the composition of OH DQA teams(6) Outline the work plan, including roles, responsibilities, and timeline for the conduct of DQAOnce approved, this DQA Plan and enhanced DQA tools will assist OH A/CORs and IPs in assessing the quality,strengths, weaknesses, and limitations of reported performance data.1.4. METHODSWe reviewed existing documents related to conducting DQAs. We reviewed documents that were producedfor USAID, IPs, or relevant programs within the Philippines Department of Health (DOH). We also brieflydiscussed available USAID/OH DQA tools with our COR. We then developed the initial DQA tools based onexisting DQA tools from USAID and the MEASURE Evaluation with a focus on the USAID Standards on DataQuality.The initial set of DQA tools was pretested in the TB Platforms office in Makati City on March 12, 2019 and inhealth facilities in Marawi City on March 5 and 7, 2019. We then revised the DQA tools based on feedback fromthe respondents. Annex A provides a brief report of the DQA pretesting activity.USAID.GOVHEALTH PORTFOLIO DATA QUALITY ASSESSMENT PLAN AND TOOLS 6

2. OVERVIEW OF DATA QUALITYASSESSMENT12.1. DEFINITION OF ACTIVITIES RELATED TO DATA QUALITYTable 3 delineates the processes and activities that are related to data quality. Additionally, this section identifieswho will lead the activities and when the activities should be conducted. This section is aligned with the USAIDADS 201.3.5.8.Table 3. Comparison of Data Quality ActivitiesData Quality Activities1. Data quality check2. Data verification3. Data validationDefinitionWho will lead/conductExamination of recordsof IPs and IP partners2 interms of completeness,accuracy, consistency,and timeliness of data.Standard component ofDQAs. Checking IPrecords for consistency(1) with data reported toUSAID, and (2) in termsof totals and datadisaggregation.An activity focusing onconfirming the validity3When to use/conductIPs’ M&E specialist/M&Eofficer/M&E assistantDepends on the M&Eplan of theorganizationUSAD staff/DQA teamPart of the DQAM&E specialist/datamanagement specialistAs neededThis section is based on the following documents: Bureau for Policy, Planning and Learning (PPL). How to note: Conduct a Data Quality Assessment. USAID (2017). conducting-data-quality-assessment-dqa MEASURE Evaluation. Data Quality Audit Tool - Guidelines for implementation.URL: ations/ms-08-29 Mumbere O. and Kopi L. How to conduct a Data Quality Assessment (DQA): An Aid Memoir for a COR/AOR. USAIDDemocratic republic of Congo (2012). MEASURE Evaluation. User Manual - Routine Data Quality Assessment Tool (2015).URL: ations/ms-17-117 Philippine Business for Social Progress (PBSP). Guidelines in conducting Data Quality Check in support of the TuberculosisControl Program (n.d.).URL: f United States Agency for International Development (USAID). Performance Monitoring & Evaluation TIPS: Data QualityStandards Number 12 2nd edition. (2009).URL: s-dataqualitystandards.pdf United States Agency for International Development (USAID). Automatic Directive System (ADS) 201.3.5.8. ments/1870/201.pdf2 IP partners are non-USAID institutions that work with the IP towards a shared goal. Examples of IP partners include but are not limitedto: health facilities, academic institutions, non-government organizations, etc.3 See subsection 2.2. for details.1USAID.GOVHEALTH PORTFOLIO DATA QUALITY ASSESSMENT PLAN AND TOOLS 7

Table 3. Comparison of Data Quality ActivitiesData Quality Activities4. DQA5a. USAIDadministered DQAb. DQA selfassessmentc. Routine DQA45DefinitionWho will lead/conductand reliability4 ofreported indicator data.Assessment of thestrengths, weaknessesand limitations ofindicators based on theUSAID Data QualityStandards.Using primarily qualitativemethods, USAIDadministered DQAactivities are carried outto assess the strengthsand weaknesses ofindicators based on theUSAID Data QualityStandards.This refers to the IP’sself-assessment of its owndata collection systems.This is an optional part ofthe USAID-administeredDQA, which aims to fullyinvolve the IPs in theDQA process.This is scheduled andconducted by the IPs toassess the quality of theirdata in terms of theUSAID Standards onData Quality and tostrengthen their dataUSAID staff/DQA teamIPs’ M&E specialistIPs M&E specialistWhen to use/conductFor new standardindicators, althoughDQA can be doneduring the process ofcollecting data, it ispreferable to do theDQA after datacollection iscompleted. It shouldbe conducted withinone (1) year beforereporting the data toUSAID/Washington.Conducted everythree (3) years at theminimum for currentstandard indicators.During the DQAactivitySpecified in the IPs’M&E planSee subsection 2.2. for details.For the purposes of this document, “DQA” refers USAID-administered data quality assessment.USAID.GOVHEALTH PORTFOLIO DATA QUALITY ASSESSMENT PLAN AND TOOLS 8

Table 3. Comparison of Data Quality ActivitiesData Quality Activities5. Data quality audit*DefinitionWho will lead/conductmanagement andreporting systems.Using formal methods,Data Quality Audits arecommissioned by thefunding agencies to assessa project/partner’s abilityto report good qualitydata.External audit teamWhen to use/conductConducted everythree to five years onpriority indicators.* The conduct of a data quality audit is beyond the scope of this DQA Plan.2.2. USAID DATA QUALITY STANDARDSTable 4 provides a brief discussion on the USAID Data Quality Standards. The DQA standards discussed in thissection are consistent with the USAID ADS 201.3.5.8.Table 4. USAID Data Quality StandardsData Quality Standard1. Validity1.1. Face validity1.2. Attribution1.3. Measurement error1.3.1. RepresentativenessUSAID.GOVDescriptionData are said to be valid if the data clearly and adequately representwhat is intended to be measured. Validity has several dimensions,namely: (1) face validity; (2) attribution; and (3) measurement error.The indicator has face validity when an expert from the field or anoutsider agrees that the data are true measure of the result. Thereis a high degree of face validity when the data are truerepresentations of the indicator and the indicator is the truemeasure of the result.There is attribution when changes in the indicator and result can beplausibly associated with the interventions of USAID and/or USAIDIPs.Measurement error results from poor research design and/orineffective implementation of the data collection methodology.Some degree of measurement error occurs despite efforts toensure that the data collection methodology is strictly followed.Management judgments in terms of the level of accuracy necessaryto make meaningful decisions and costs must be the basis of thejudgments about the acceptable measurement error.Measurement error has two components: (1) Sampling error orrepresentativeness, and (2) non-sampling error.Data is said to be representative when the sample collected reflectsthe target population. To ensure representativeness, sample datamust be derived from a sampling design and sample size appropriateto the target population.HEALTH PORTFOLIO DATA QUALITY ASSESSMENT PLAN AND TOOLS 9

Table 4. USAID Data Quality StandardsData Quality Standard1.3.2. Non-sampling error2. Reliability3. Timeliness4. Precision5

strengthen health behaviors; (2) fortify quality of health services; and (3) institutionalize key health systems (see Figure 1). The Health Portfolio focuses on the thematic areas of tuberculosis (TB), family planning (FP), adolescent and youth reproductive health (AYRH), and health systems strengthening (HSS). The OH selected project sites