Metadata Management As A Key Component To Data Governance, Data .

Transcription

Metadata Management as a Key Component to DataGovernance, Data Stewardship, and Data Quality ManagementWednesday, July 20th 2016

Confidential, Datasource Consulting, LLC2

Multi-Domain Master Data Management delivers practicalguidance and specific instructions to help guide planners andpractitioners through the challenges of a multi-domain masterdata management (MDM) implementation. Authors Mark Allenand Dalton Cervo bring their expertise to you in the onlyreference you need to help your organization take master datamanagement to the next level by incorporating it across multipledomains.Morgan Kaufmann, Elsevier, April 2015Authors Dalton Cervo and Mark Allen show you how toimplement Master Data Management (MDM) within yourbusiness model to create a more quality controlled approach.Focusing on techniques that can improve data qualitymanagement, lower data maintenance costs, reduce corporateand compliance risks, and drive increased efficiency in customerdata management practices, the book will guide you insuccessfully managing and maintaining your customer masterdata.John Wiley & Sons, May 2011Confidential, Datasource Consulting, LLC3

Agenda A Customer’s Story Metadata and Metadata Management Metadata Management / Governance Metadata Management / Stewardship Metadata Management / Data Quality Management Conclusion

A CUSTOMER’S STORYConfidential, Datasource Consulting, LLC5

A Customer’s Story Problem: what’s the enterprise definition of Net Charge Off (NCO), how is itcalculated, and who uses it?Generic definition of Net Charge Off (from Investopedia):– A net charge off (NCO) is the dollar amount representing the difference between grosscharge-offs and any subsequent recoveries of delinquent debt. Net charge offs refer to debtowed to a company that is unlikely to be recovered by that company. This "bad debt" oftenwritten off and classified as gross charge-offs. If, at a later date, some money is recovered onthe debt, the amount is subtracted from the gross charge-offs to compute the net charge-offvalue.Challenges:––––––The many interpretations by: accounting, collections, risk, corporate planning, andremarketingIdentify sources of relevant dataDetermine calculations utilizedEstimate actual value of the assetHow to compute expenses and recoveries after charge offImpact of accounting time periodConfidential, Datasource Consulting, LLC6

METADATA AND METADATAMANAGEMENTConfidential, Datasource Consulting, LLC7

What’s Metadata?It’s more than just “data about data”“Metadata is structured information thatdescribes, explains, locates, or otherwisemakes it easier to retrieve, use, or managean information resource.”NISO – National Information Standards OrganizationConfidential, Datasource Consulting, LLC8

Metadata CategoriesBusiness Metadata Business definitions Business rules, regulations, and data quality expectationsTechnical Metadata Physical data structures and interfaces Documentation for auditing derivations, dependencies, and data flowOperational Metadata Statistics about data movement: frequency, record counts, component by componentanalysis and other statisticsConfidential, Datasource Consulting, LLC9

Islands of MetadataConfidential, Datasource Consulting, LLC10

Metadata Across the OrganizationMetadata Repository Enterprise Business GlossaryEnterprise Process GlossaryLOB’s Business GlossaryLOB’s Business Process GlossaryConceptual ModelsLogical ModelsPhysical ModelsPhysical StructuresData DictionaryData LineageInterface InformationData TransformationsBatch Job DescriptionsData Movement StatisticsData Security RulesReport and Correspondence MappingConfidential, Datasource Consulting, LLC11

Metadata ManagementConfidential, Datasource Consulting, LLC12

Collecting MetadataBusiness TrackBusiness MetadataTechnical MetadataTechnical TrackOperational MetadataConfidential, Datasource Consulting, LLC13

Distributing MetadataTechnical MetadataBusiness TrackBusiness MetadataTechnical TrackOperational MetadataConfidential, Datasource Consulting, LLC14

Managing MetadataData LifecycleManagementData ess RuleManagementData QualityRuleManagementOrganizing, Categorizing,Approving, Maintaining,& FacilitatingData SecurityManagementRisk MitigationAudit TrailDataTransformationImpact AnalysisConfidential, Datasource Consulting, LLC15

Technology in Metadata Management Why is technology important?Considerations when selecting a vendorBusiness/IT considerationsAdoption, implementation, and maintenance challenges Some of the players:Confidential, Datasource Consulting, LLC16

Metadata Management &DATA GOVERNANCEConfidential, Datasource Consulting, LLC17

There are varying perspectives on DataGovernance The exercise of authority, control andshared decision-making (planning,monitoring and enforcement) over themanagement of data assets. DataGovernance is high-level planning andcontrol over data management.DAMA is a control that ensuresthat the data entry by anoperations team member orby an automated processmeets precise standards,such as a business rule, adata definition and dataintegrity constraints in thedata model. the specification of decision rights and anaccountability framework to ensureappropriate behavior in the valuation, creation,storage, use, archiving and deletion ofinformation. It includes the processes, rolesand policies, standards and metrics thatensure the effective and efficient use ofinformation in enabling an organization toachieve its goals.Wikipedia unites people, process,and technology to changethe way data assets areacquired, managed,maintained, transformed intoinformation, shared acrossthe company as commonknowledge, and consistentlyleveraged by the business toimprove profitability. is a system of decision rights andaccountabilities for information-relatedprocesses, executed according to agreedupon models which describe who can takewhat actions with what information, andwhen, under what circumstances, usingwhat methods.DG InstituteGartner refers to the overall management of theavailability, usability, integrity, and security of thedata employed in an enterprise.Tech TargetThe execution and enforcement of authority overthe management of data assets and theperformance of data functions.Chris DegerTDANConfidential, Datasource Consulting, LLC18

The Datasource Definition ofData Governance (DG) Datasource uses Data Governance as an umbrella concept to cover thedisciplines often referred to as Data Governance (DG) and DataManagement (DM). From a DG perspective, it defines who in the organization gets to makewhat decisions about what data and establishes process and structure tosupport that governance. From a DM perspective, it facilitates and coordinates the myriad ofenterprise functions and organizations, processes and technologies to bringabout data value optimization. Where there are gaps in realizing data value optimization, Data Governanceworks with the organization to fill them.Confidential, Datasource Consulting, LLC19

Diminishing Value of Your Data, TodayIneffective InterdepartmentalCommunication Conceals reliance on common data Ramps up redundant work Increases shadow IT data costs Magnifies management confusionInsufficient Integration / DesktopIntegration Increases data system costs Increases shadow IT data costs Atrophies operational agility Hampers enterprise perspectiveData Quality Issues & Perceptions Breaks systems & processes Impacts analytics & reporting Tramples data trust Atrophies operational agility Breeds bad business decisionsInformation Security Confusion Compounds compliance risk Raises data security risk Hampers data access Limits performance managementprospectsDeficient Data Change Control Breaks systems & processes Impacts analytics & reporting Tramples data trustBattling Business Rules Increases compliance risk Tramples data trust Creates reports / analysis conflicts Ramps up redundant work Magnifies management confusionConfidential, Datasource Consulting, LLC20

Diminishing Value of Your Data, TodayAnalytic Group Fragmentation &Regulatory Disconnects Creates reports / analysis conflicts Magnifies management confusion Compounds compliance risk Tramples data trust Ramps up redundant work Limits performance managementprospectsLack of Common Names, Definitions,Context / Metadata Magnifies management confusion Ramps up redundant work Atrophies operational agility Breeds bad business decisions Complicates communication Compounds compliance risk Tramples data trust Creates reports / analysis conflictsInadequate Data Accountability Complicates communication Compounds compliance risk Raises data security risk Magnifies management confusion Promotes non-productive work Atrophies operational agility Tramples data trust Conceals reliance on common dataConfidential, Datasource Consulting, LLC21

These are just some of the more common issues that diminish the value of your data– you are dealing with at least a few of them now. Restoring data value requires datagovernance orchestration across these issues, and beyond.Confidential, Datasource Consulting, LLC22

In part, DG efforts can fail for the samereason any organization program fails Failure to establish andcommunicate a compelling visionPoor ongoing communication andselling of the programPoor planningPicked wrong success measuresPolitical naïvetéUnmanaged expectationsResistant cultureLack of leadershipPoorly defined objectivesEtc.Lack of support (executive &otherwise)Confidential, Datasource Consulting, LLC23

but there are additional common reasonsDG programs fail“Overhead” perceptionPurely IT program positioningRaised visibility too earlyFailure to build organizational alliancesNot positioned as a business enablerLeader too juniorRun by a data person who is not a strong people personVelocity expectationsConfidential, Datasource Consulting, LLC24

DG should optimize the value of data to theorganizationTo optimize data value we can lower data costs and / orincrease data worthLower Data-Related CostsIncrease Data Worth Reduce redundant data-related workRationalize data applicationsRationalize data vendor relationshipsManage data retentionReduce risk-related costsCreate shared understanding of dataEnsure data qualityEnsure data timelinessEstablish trust in dataMake finding right data easierConfidential, Datasource Consulting, LLC25

In DG, consider the value of being able toanswer these questions Where can I find the information I need? What does this data mean? Is this data good enough for my needs? What am I allowed to do with this data? Who can help me if I have questions about this data?Confidential, Datasource Consulting, LLC26

Metadata Management & DGDiminishing Value of Data in needof DG Lack of common names, definitions,and context Inadequate data accountability Inconsistent and undefined businessand data quality rules Information security confusion Deficient data change control Ineffective communication Unidentified source of dataCollectManageDistributeMetadata Management Business GlossaryBusiness Process GlossaryBusiness Rule ManagementRisk ManagementRules and RegulationsData Lifecycle ManagementData Ownership ManagementData Quality Rule ManagementData Security ManagementImpact AnalysisAudit TrailConfidential, Datasource Consulting, LLC27

Metadata Management &DATA STEWARDSHIPConfidential, Datasource Consulting, LLC28

What is Data Stewardship? Data stewardship encompasses the tactical management and oversight ofthe company’s data assetsData stewardship is generally a business function facilitating thecollaboration between business and IT, and driving the correction of dataissuesSeveral models for Data Stewardship:–––––By Domain or EntityBy Business FunctionBy SystemBy Business ProcessBy ProjectDataStewardshipDataGovernanceData QualityManagementConfidential, Datasource Consulting, LLC29

Metadata Management & StewardshipData Steward ResponsibilitiesStrategy &PlanningProjectScoping &AnalysisProjectDevelop, Test& DeployDailyOperations Understand strategy as relates to data area Define &recommend data enhancement projects Provide feedback on SBL standardsCollectManageDistribute Help Identify data sources Review & provide feedback on project Support development of test planMetadata Management Define DQ needs & make sure is integrated intobusiness requirements Help develop training on data Monitor business changes for data impact Monitor DQ metrics and recommend correctiveaction, as needed Field questions about data area of responsibility Business DefinitionsBusiness Process DefinitionsBusiness RulesRules and RegulationsData Lifecycle, including lineage andtransformationsData Quality RulesData Security RulesData DictionaryContext DefinitionsImpact AnalysisConfidential, Datasource Consulting, LLC30

Metadata Management &DATA QUALITY MANAGEMENT

What is Data Quality Management? Data Quality Management (DQM) is about employing processes, methods, andtechnologies to ensure the quality of the data meets specific business requirements Trusted data delivered in a timely manner is the ultimate goal DQM can be reactive or preventive. More mature companies are capable ofanticipating data issues and prepare for them (that’s where Metadata Management iskey) DQM encompasses many activities:Data ProfilingData ValidationData Cleansing or ScrubbingData ConsolidationData MatchingData SurvivorshipData StandardizationData ReconciliationData EnrichmentData MonitoringData Quality DashboardsData Lineage and TraceabilityData Classification or CategorizationConfidential, Datasource Consulting, LLC32

Datasource DQM ProcessConfidential, Datasource Consulting, LLC33

A Look into Data Profiling (1 of 3)RequirementsData Profile Metric ResultsTypically, organizations approach Data Profiling as a 2D activity:Data Profile TechniquesConfidential, Datasource Consulting, LLC34

A Look into Data Profiling (2 of 3)But Data Profiling is in a Spectrum:Confidential, Datasource Consulting, LLC35

A Look into Data Profiling (3 of 3)Therefore, a 3rd dimension must be considered, which is tied tometadata management:Data ProfileMetric ResultsRequirementsData ProfileTechniquesMetadataInformationConfidential, Datasource Consulting, LLC36

Metadata Management & DQMBesides Data Profiling, the other DQMactivities can certainly benefit fromMetadata Management:-Data ValidationData Cleansing or ScrubbingData ConsolidationData MatchingData SurvivorshipData StandardizationData ReconciliationData EnrichmentData MonitoringData Quality DashboardsData Lineage and TraceabilityData Classification or CategorizationCollectManageDistributeMetadata Management Business DefinitionsBusiness Process DefinitionsBusiness RulesRules and RegulationsData Lifecycle, including lineage andtransformationsData Quality RulesData Security RulesData DictionaryContext DefinitionsImpact AnalysisConfidential, Datasource Consulting, LLC37

CONCLUSIONConfidential, Datasource Consulting, LLC38

Collaborative Data ManagementMetadata ManagementData GovernanceData Quality ManagementData StewardshipConfidential, Datasource Consulting, LLC39

A Customer’s Story – ConclusionMetadata Management Business definitions– Accounting NCO– Operational NCO Business and DQ rules aroundNCO Sources of related data, attributesand their lineage/transformations NCO usage by different businessprocesses and reportsData Quality Management Monitoring of NCO quality Dashboards and scorecards Alerts on suspicious accounts Reports on company performanceand risk levelData Governance Business alignmentEnterprise standardsRisk mitigationDispute resolutionImpact analysisOwnership assignmentData Stewardship Easy identification ofsources/attributes related to NCO Proactive monitoring of thresholdsand expected values Expedited issue resolution Streamlined maintenanceConfidential, Datasource Consulting, LLC40

Datasource ConsultingWe are a consulting company that focuses exclusively on Enterprise DataManagement & Business Intelligence, including both strategic andimplementation services. We are passionate about data.STRATEGICIMPLEMENTATIONData Governance (People, Process, Tech)Roadmaps & Assessments (BI, DW, EDM)Program Management, Project PlansVendor Tool SelectionData Architecture, Data IntegrationData Quality, Business IntelligenceMaster Data ManagementReporting & Analytics, Cloud100% Success Rate 80% of clients stay with Datasource over multiple years.dcervo@datasourceconsulting.com

Data Governance (DG) Datasource uses Data Governance as an umbrella concept to cover the disciplines often referred to as Data Governance (DG) and Data Management (DM). From a DG perspective, it defines who in the organization gets to make what decisions about what data and establishes process and structure to support that governance.