Transcription
InformationSystems for Health(IS4H)TOOLKITIS4H MaturityAssessment toolDepartment of Evidence and Intelligence for Action in HealthPAHO/WHO
1Maturity Assessment tool – Version 2.0Information Systems for Health ToolkitIS4H Maturity Assessment toolIS4H- MM 2.0AcknowledgmentsThis work would not have been possible without the technical and financial support ofthe United States Agency for International Development (USAID),the Government of Canada and the International Development Bank (IDB)Version 2.0 – May, 2019Department of Evidence and Intelligence for Action in HealthPan-American Health Organization - World Health Organization1 Page
2Maturity Assessment tool – Version 2.0Toolkit metadataNameIS4H Maturity Assessment toolAcronymIS4H – MMVersion2.0DateMay, 2019DescriptionThe Information Systems for Health Maturity Assessment Tool (IS4H-MM)describes the method, tool and questions for assessing organizational capacityrelated to governance, data management, digital transformation, innovation andknowledge managementResponsibleentityDepartment of Evidence and Intelligence for Action in Health (EIH)Entity DirectorJacobo Finkelman, Director EIH, a.i.Focal pointMarcelo D’Agostino, Senior Advisor Information Systems for Health, PAHOUrlhttp://www.paho.org/ishContributors Daniel Doane, International Expert, ArgentinaMyrna Marti, International Expert, ArgentinaIan Brooks, University of IllinoisPatricia Ruiz, Advisor, Information Systems and Health Analysis, PAHOAndrea Gerger, Advisor, Information Systems at PAHOJennifer Nelson, Advisor, International Development BankPartners Caribbean Technical Working group for IS4HUnited States Agency for International Development (USAID)Government of CanadaInternational Development Bank (IDB)Piloted by / andoriginated with Ministry of Health, GuyanaMinistry of Health, JamaicaMinistry of Health, SurinameMinistry of Health, EcuadorMinistry of Health ParaguayMinistry of Health, Government of the Virgin Island2 Page
Maturity Assessment tool – Version 2.03Table of ContentsIntroduction .4IS4H-MM Conceptual framework .5IS4H-MM Description .6Data Management and Information Technologies (DMIT) .6DMIT – Components .6DMIT – Characteristics.7DMIT – Assessment (key) questions .9Management and Governance (MAGO) .17MAGO – Components .17MAGO – Characteristics .18MAGO – Assessment (key) questions .21Knowledge Management and Sharing (KMSH) .28KMSH – Components .28KMSH – Characteristics .29KMSH – Assessment (key) questions .31Innovation (INNO) .37INNO – Components .37INNO – Characteristics .38INNO – Assessment (key) questions.41Assessment methodology .47Outputs and deliverables .473 Page
Maturity Assessment tool – Version 2.04IntroductionThe Information Systems for Health Maturity Assessment Tool (IS4H-MM) describes themethod, tool and questions for assessing organizational capacity related to governance,data management, digital transformation, innovation and knowledge management. TheIS4H-MM is part of the IS4H Blueprint for a successful implementation (Figure 1),and is organized according to the 4 strategic goals of the IS4H conceptual Framework.Figure 1. IS4H Blueprint for a successful implementation4 Page
Maturity Assessment tool – Version 2.05IS4H-MM Conceptual frameworkThe IS4H-MM is also a reference framework guiding Information Systems for Health tokeep walking through the changing path of information and knowledge revolution, andhow countries and organizations might grow in capabilities to operate, interact and benefitfrom them. The diagram below illustrates the five levels of maturity (Figure 2).Figure 2. Maturity Assessment Tool (Conceptual framework)Assessment of the five progressive levels of IS4H-MM provides the awareness forplanning where to go by Information Systems plans and roadmaps.Maturity is assessed against key capability characteristics for each strategic goal at eachlevel of the Maturity Model. It is possible for an organization to demonstrate differentmaturity levels within each strategic goal.Maturity Level characteristics and components are detailed below. They are color codedaccording to each strategic goal of the framework that are reflected as DMIT, MAHO,KMSH and INPE in Figure 2).5 Page
Maturity Assessment tool – Version 2.06IS4H-MM DescriptionData Management and InformationTechnologies (DMIT)DMIT – Components Data Sources: Data collection mechanisms and technologies. Structured data refersto content that has a predefined structure and is normally classified and stored in atraditional relational database. Unstructured data refers to different types of contentthat that is not classified in a standard manner Information Products: Health data that are processed and published openly in avariety of formats that accomplish the different needs of IS4H constituencies.6 Page
7Maturity Assessment tool – Version 2.0 Standards for Quality and Interoperability: Use and availability of data standards,identifiers, standards for interoperability and a national health informationarchitecture. Data Governances: Health data governance is the framework for establishing subregional and national strategies, objectives, policies, standards, and tools for themanagement of technical data, which is supported by a legal framework IT Infrastructure: Availability and maintenance of Tools, Networks, Hardware andSoftware to support IS4H. Interoperability among platforms and integration of datarepositories.DMIT – CharacteristicsMaturity Level characteristicsLEVEL 1LEVEL 2IS4H FrameworkComponent Key data sources (e.g., vital statistics, demographics,morbidity and surveillance data) have been identified.Data Sources Data are routinely collected from key data sources, butdata flow may be ad hoc or have challenges that impactquality.Data Sources Some data are collected electronically, but paper-baseddata collection is still common.Data Sources Some information products are generated, but notroutinely and require intensive work and use of resources.Information Products Data is not readily shared across units, with stakeholdersor with the public.Information Products Sharing data frequently requires permission from seniorlevels.Information Products There are few formal data standards enforced; evidence ofsome use of data standards (e.g. ICD-10; standardizedindicators, etc.)Standards for Qualityand Interoperability Data is not readily available for other purposes oncecollected and published in an information product.Data Governance Basic IT tools are generally available (e.g., hardware,software, internet connectivity available as required bystaff), but may be older or not performing well.IT Infrastructure Data are routinely collected electronically from all keysources.Data Sources Information products are efficiently and routinely produced. Information Products Some identifiers are used in individual data sources, butare not consistent or available across data sourcesStandards for Qualityand Interoperability7 Page
8Maturity Assessment tool – Version 2.0LEVEL 3LEVEL 4LEVEL 5 Standards for interoperability have been identified.Standards for Qualityand Interoperability Some formally documented health data managementprocesses and best practices are available (e.g., dataquality frameworks, data standards, policies, SOPs, etc.).Data Governance Core datasets are readily available.Data Governance Data are rarely integrated for analysis across varioussources.Data Governance Some metadata are documented and maintained (e.g.indicator compendiums, data dictionaries).Data Governance Appropriate IT infrastructure (networks, servers, etc.) andtools (hardware, software) are widely available andgenerally reliable.IT Infrastructure Information products are routinely produced that meet thespecific needs of various stakeholders.Information Products Standards for interoperability have been formally adopted,and the national health information architecture has beenformally defined and documented.Standards for Qualityand Interoperability Health data management policies, procedures and bestpractices are consistently applied, resulting in theavailability of quality data.Data Governance Formal data governance mechanisms (committees,policies, data quality frameworks, data sharingagreements, etc.) have been established among thenational health authorities, and are functioning effectively.Data Governance There are continuous improvement processes establishedto monitor and invest in data quality.Data Governance There is evidence of interoperability between some healthinformation platforms.IT Infrastructure Formal health data governance mechanisms have beenestablished at the national level with stakeholders outsidethe national health authority.Data Governance A national identifier is available for integrating health datafrom all sources.Standards for Qualityand Interoperability There is evidence of significant interoperability acrosshealth information platforms.IT Infrastructure Integrated national data repositories from multiple datasources.IT infrastructure Data from multiple data source types, includingunstructured sources such as social media and varioustypes of devices (e.g., Internet of Things – IoT) are used inhealth analysis.Data Sources8 Page
9Maturity Assessment tool – Version 2.0 Large data sets integrated from multiple sources arereadily available for analysis to support decision-making.Data Sources Health information systems are interoperable, enabled bya national infrastructure that uses current standards,technologies, and architectures.Standards for Qualityand InteroperabilityDMIT – Assessment (key) questionsDMITDATA MANAGEMENT AND INFORMATION TECHNOLOGIES - QUESTIONSDATA SOURCES1.What are the country key data sources:2. Vital statistics Mortality Morbidity Chronical diseases NCD Immunization HIV Maternal health other .For key data sources identified data is collected: :3. On demand On a regular basis Don t collectFor each key data source, identify if data is collected by: Paper FormsTally sheetsMicrosoft ExcelDatabaseSpecify: Access, Oracle, MySQl, etc. EMRSpecify:Other softwareSpecify: A census was carried out in the past: 0-3 years3- 5 years5-8 years8-10 years9 Page
10Maturity Assessment tool – Version 2.0DMITDATA MANAGEMENT AND INFORMATION TECHNOLOGIES - QUESTIONS4.Is there a routine frequency of data collection for the identified key data sources?5. Real Time Daily Weekly 1 month 1-6 months 6-12 months more than 12 months On demandData is collected in6.7. structured format not in a structured format Mix of structured and unstructured formatsData collected include demographic disaggregated units (e.g. age, sex, residence by smallestadministrative level). Yes Some No N/AThe health-facility staff have adequate capacity to implement data collection;STANDARDS FOR QUALITY & INTEROPERABILITY8.Health data standards adopted (classification systems, data standards, Medical terminologies)9. No Under implementation YesCommon data model implemented No Under development Developed but not implemented Under implementation Yes – routinely adoptedDATA GOVERNANCE10.The health-facility staff adopts procedures to:11. select health data collections, process the data with data quality routines analyze the data from various sources in health None of the above?Health metadata (data about data) documented12. No Some documentation YesMetadata best practices have been defined and adopted No10 P a g e
11Maturity Assessment tool – Version 2.0DMITDATA MANAGEMENT AND INFORMATION TECHNOLOGIES - QUESTIONS13. Under development Developed but not implemented Under implementation Yes – routinely adoptedDo executives, employees or stakeholders understand the types and value of metadata?14. Yes No SomeThere is a written set of procedures for data storage and cleaning15. no procedures some procedures Procedures and policies documented but not fully implemented Procedures and policies documented and implemented at the region and national levels Procedures and policies documented and implemented at the facility, region and national levelsThere is a written set of procedures for data collection16. no some Developed but not fully implemented Developed and implemented at the region and national levels Implemented at the facility, region and national levelsThe organization has implemented procedures that define how health data sets are to be used byauthorized personnel17. no some Developed but not fully implemented Developed and implemented at the region and national levels Implemented at the facility, region and national levelsThe organization has implemented a formal Data Governance mechanism?18. no some Developed but not fully implemented Developed and implemented at the region and national levels Implemented at the facility, region and national levelsThe organization has a set of rules and regulations in support to health data management?19. no some Developed but not fully implemented Developed and implemented at the region and national levels Implemented at the facility, region and national levelsStakeholders are aware of the specific Data Governance capabilities that are available at the organization NoSomeYesN/A11 P a g e
12Maturity Assessment tool – Version 2.0DMITDATA MANAGEMENT AND INFORMATION TECHNOLOGIES - QUESTIONS20.The staff is aware of the purpose or value of the Data Governance program21. No Some Yes N/ADatasets from major health data sources are consistent and have data quality measures.22.23. Yes No In processHealth institutions adopts unique identifier codes to facilitate the merging of multiple databases fromdifferent health data sources. Yes No In processCompleteness and precision data quality checks on all data may be performed at the point of entry foreach mandatory attribute from each source system.24.Conformity checks and integrity checks on data are recent and have coverage of major data sources.25.Perform health data consistency and health data quality policies are:26. Non-existent Under development Developed but not in place In place.There are measurement cycles for health data consistency and quality activities27. No In process Yes, recently adopted Yes, and it has been done in the past 3 yearsGlobal Positioning System (GPS) coordinates for each health facility are included in the database28. No In process YesThere is a written set of procedures for data quality control29. No some procedures Procedures and policies documented but not fully implemented Procedures and policies documented and implemented at the region and national levels Procedures and policies documented and implemented at the facility, region and national levelsThere is a process for measuring health data quality NoUnder developmentDeveloped but not implementedUnder implementationYes – routinely adopted12 P a g e
13Maturity Assessment tool – Version 2.0DMITDATA MANAGEMENT AND INFORMATION TECHNOLOGIES - QUESTIONS30.31.32.Have data quality best practices been defined and adopted as official organizational data policies? No Under development Developed but not implemented Under implementation Yes – routinely adoptedQuality checks on all data performed at the point of entry for each mandatory attribute from each sourcesystem. No Under development Developed but not implemented Under implementation Yes – routinely adoptedThe national core data set has been defined No In process YesINFORMATION PRODUCTS33.34.35.36.There is a written set of procedures for data presentation for target audiences. no procedures some procedures Procedures and policies documented but not fully implemented Procedures and policies documented and implemented at the region and national levels Procedures and policies documented and implemented at the facility, region and national levelsThe organization has implemented a catalog of types of Information Product (e.g. databases, reports,graphical material, publications, infographics etc) No Ad Hoc Yes N/AThe organization provides graphs to display information at subnational health administrative offices (e.g.,regional/provincial, district) and health facilities No Ad Hoc Yes N/AThe organization has implemented specific rules and procedures for staff to report, explain or justifyactions related to the use of health data managed by the institution no proceduressome proceduresProcedures and policies documented but not fully implementedProcedures and policies documented and implemented at the region and national levelsProcedures and policies documented and implemented at the facility, region and national levels13 P a g e
14Maturity Assessment tool – Version 2.0DMITDATA MANAGEMENT AND INFORMATION TECHNOLOGIES - QUESTIONS37.Accurate population projections by age and sex are available for small areas (districts or below) for thecurrent year38.Published statistics from civil registration system are disaggregated by: (1) sex; (2) age; and (3)geographical or administrative region (or urban/rural)39.Lag between the time that data were collected and the time that statistics from civil registration/SRS/DSSwere published40. Real Time 1 Day 1 Week 1 month 1-6 months 6-12 monts more than 12 months On demandThere is a common data presentation format41. No In process Yes N/AInformation products are developed:42. Routinely On demand Not producedProduction of information products is:43. Automated (systems delivered) Mix of automated and manual ManualInformation products meet the needs of: Central Level Central and local level Various stakeholders.IT INFRASTRUCTURE44.The organization has a formal mechanism for the adoption of ICT standards45. no some Developed but not fully implemented Developed and implemented at the region and national levels Implemented at the facility, region and national levelsThe technology performs as it was intended46. No Some YesThe technology follows generally accepted industry standards14 P a g e
15Maturity Assessment tool – Version 2.0DMITDATA MANAGEMENT AND INFORMATION TECHNOLOGIES - QUESTIONS47.48. No Some YesThere is equipment for maintaining and updating the database and maps on health facilities and services: Yes Some facilities have NoThe MOH/Organization achieved convergence of interconnected and interoperable health informationsystems.49. No Under development Developed but not implemented Under implementation Yes – routinely adoptedData exchange services implemented50. No Under development Developed but not implemented Under implementation Yes – routinely adoptedDirectory services in place51. No Under development Developed but not implemented Under implementation Yes – routinely adoptedThere are norms and standards for software development52. No Under development Developed but not implemented Under implementation Yes – routinely adoptedReporting of security incidents is done:53. Instantly 2 to 12 hours delay 12 hours to 1 day delay During the week More than a week later Not reportedInvestigating of security incidents is done: Instantly2 to 12 hours delay12 hours to 1 day delayDuring the week15 P a g e
16Maturity Assessment tool – Version 2.0DMITDATA MANAGEMENT AND INFORMATION TECHNOLOGIES - QUESTIONS54. More than a week later Not investigatedThe organization has a dedicated budget for ICT Infrastructure (Computers, Network, InternetConnection, etc)16 P a g e
Maturity Assessment tool – Version 2.017Management and Governance (MAGO)MAGO – Components Leadership and Coordination: Coordination and distribution of the governancestructure for IS4H accountability and decision-making at the managerial and technicallevel among all actors. Strategic and Operational Plans: Addressing IS4H under policies, strategies andSOPs at the national, regional and local level. Mechanisms for developing or adoptingan IS4H governance strategy or policy that promotes a better decision- and informedpolicy-making mechanisms Organizational Structures and Functions: Organizational Structure & Informationflows of health-related institutions. Roles and responsibilities IS4H health systemactors.Financial Resources: Budget for IS4H implementation, sustainability, investment.Resources mobilization plans and ERP systems. 17 P a g e
18Maturity Assessment tool – Version 2.0 Human Resources: Human capital for planning, implementing, and managing IS4H.Competency building activities to strengthen to IS4H skills. Job functions identified toeffectively support IS4H. Multisectoral Collaboration: identification, engagement and coordination of the keyactors/stakeholders for IS4H, including public–private relationship. Legislation, policy and compliance: Key and core legislation, policy andcompliance mechanisms, elements to enable IS4H implementation, operation andmaintenance. National and international agreements: National and International agreements tocontextualize national plans and investments. Commitment to regional and globalmandates.MAGO – CharacteristicsMaturity Level CharacteristicsLEVEL 1IS4H FrameworkComponent Accountability and decision-making for IS4H (e.g., datamanagement, health analysis, information technology, etc.) isdistributed across different units within national healthauthorities, and investments and activities are typically notcoordinated.Leadership andCoordination Components of IS4H are partially addressed under thenational health strategy or related strategic plans, but there isno single integrated strategic or operational plan for IS4H.Strategic andOperational Plans There are gaps in IS4H services or functions, and/or servicesand functions may be duplicated across units/programs.OrganizationalStructures andFunctions Success depends on knowledgeable individuals within theorganization rather that on repeatable processes documentedin unit descriptions, job descriptions, policies and SOPs.OrganizationalStructures andFunctions There are identified human resource constraints for planning,implementing, and managing IS4H, but there is no formal planfor addressing human resource needs.HumanResources IS4H activities/resources are not formally identified inprogram/unit budgets.FinancialResources While it is sometimes possible to secure one-time financialresources for IS4H investments, required investments aredifficult to sustain.FinancialResources Key stakeholders within the health sector or closely relatedsectors (e.g., vital statistics, national statistics, internationalMultisectoralCollaboration18 P a g e
19Maturity Assessment tool – Version 2.0donors) have been identified, but engagement andcoordination are typically ad hoc.LEVEL 2 There is general awareness that there are gaps in legislation,policy and compliance mechanisms that create barriers to theeffective use of IS4H, but specific gaps and needs have notbeen formally documented.Legislation Policyand Compliance There is some awareness of obligations under national andinternational agreements (e.g., IHR, SDGs, PAHO CoreIndicators, national statistical reporting requirements, etc.)National andInternationalAgreements IS4H investment decisions are coordinated at theLeadership andmanagement level within individual national health authoritiesCoordination(e.g., MOH, reginal health authorities, health facilities, etc.) butnot formally coordinated among health authorities or othernational actors. IS4H activities are identified in individual unit/program annualoperational plans of national health authorities, but are notintegrated or aligned across units/programs.Strategic andOperational Plans Accountability and responsibility for IS4H functions withinnational health authorities have been defined, and there areplans in place for organizational restructuring or re-alignmentto rationalize functions and decision-making.OrganizationalStructures andFunctions Skills and job functions required to effectively support IS4Hhave been identified, although not all resources have yet beensecured.HumanResources There is evidence of competency building activities (training,workshops, conferences) for IS4H domains but these aretypically ad hoc.HumanResources IS4H activities are identified in individual unit/program annualFinancialbudgets of national health authorities, but are not integrated or Resourcesaligned across units/programs. The financial resources requirements to effectively sustainIS4H have been identified, but not fully secured withinoperational budgets.FinancialResources The roles and responsibilities of all the actors in the nationalhealth system are well documented.MultisectoralCollaboration Informal relationships have been established with keymultisectoral national actors, including private sectororganizations. These national actors frequently participate ininformal information sharing and planning activities.MultisectoralCollaboration Requirements for IS4H enabling legislation, policy andcompliance mechanism have been identified, but not yetimplemented.Legislation Policyand Compliance19 P a g e
20Maturity Assessment tool – Version 2.0LEVEL 3LEVEL 4 There are policies and SOPs that address ethical use andprotection of health data (e.g., privacy, security, secondaryuse), but there may be gaps in regulation or legislation.Legislation Policyand Compliance Obligations under national and international agreements arefrequently met, but with high resource impact.National andInternationalAgreements There is a formal governance structure in place for strategicplanning and oversight of IS4H among the national healthauthorities (e.g. MOH, regional health authorities, healthfacilities, etc.).Leadership andCoordination There is a formal strategic plan in place among national healthauthorities for strengthening IS4H that reflects the IS4HStrategic Framework.Strategic andOperational Plans Operational plans of the units within national health authoritiesreflect IS4H activities and outcomes based on the IS4HStrategic Plan.Strategic andOperational Plans An organizational structure that defines clear accountabilitiesand responsibilities for IS4H has been fully implementedwithin/among national health authorities, and is reflected inunit/program mandates and job descriptions.OrganizationalStructures andFunctions Enough human resources with the required skills to effectivelyimplement and sustain IS4H have been secured.HumanResources Relevant IS4H skills and competency development areintegrated into training plans for leadership, management andstaff.HumanResources There is a plan in place for resource mobilization for specificIS4H capital investments, and financial resources required forthe sustainable implementation and operations of IS4H havebeen secured with annual budgets.FinancialResources Formal relationships have been established with multisectoralactors, including the private sector. There are examples arecollaborative initiatives between multisector partners.MultisectoralCollaboration Legislation/regulation addressing electronic health records,privacy and secondary use of data are in place.Legislation Policyand Compliance Obligations under national and international agreements areconsistently met with an effective use of resources.National andInternationalAgreements IS4H governance structures are established at the nationallevel,Leadership andCoordination IS4H governance includes representation from 0 P a g e
21Maturity Assessment tool – Version 2.0LEVEL 5 IS4H strategic plan is established at the national level.Strategic andOperational Plans An IS4H investment framework is established at the nationallevel.FinancialResources IS4H roles, responsibilities and functions are aligned acrossmultisectoral partners.MultisectoralCollaboration Agreements enable data and information sharing acrossnational and international stakeholders.National andI
Data Governance Core datasets are readily available. Data Governance Data are rarely integrated for analysis across various sources. Data Governance Some metadata are documented and maintained (e.g. indicator compendiums, data dictionaries). Data Governance Appropriate IT infrastructure (networks, servers, etc.) and
