Transcription
Metadata Management Tutorial – Web Portal & Data Governance Using erwin MetadataManagement (EMM)MetadataManagementTutorialWeb Portal & Data GovernanceUsing erwin Metadata Management(EMM)P a g e 1 220C o p y r i g h t 2 0 1 8 ,e r w i n I n c . a n d / o r i t s a f f i l i a t e s .V e r s i o n 2 0 1 8 R 1 ( 1 0 / 1 6 / 2 0 1 8 )A l lR i g h t sR e s e r v e d .
Metadata Management Tutorial – Web Portal & Data Governance Using erwin MetadataManagement (EMM)TABLE OF CONTENTS12Introduction 51.1Conventions used in the tutorial 61.2Setting Up the Tutorial 7About Data Governance and Metadata Management tify data stores & associated design tools 14Understanding This Tutorial’s Enterprise Architecture 16Catalog the Operational Data Stores 16Catalog the Data Processes. 18Harvesting the Finance System Metadata 20Harvest the Accounts Payable model 20Harvest the Accounts Receivable model 334.1Configuration Management 374.2Populate the Published Configuration 384.34.3.14.46From Data Stores and Data Processes To Enterprise Architecture 14Configuration Management 364.2.14.2.24.2.34.2.44.2.55Data and metadata are everywhere! 10MITI’s strategy to support data and metadata management 11Metadata Harvesting 133.14General data and metadata management discussion 10Bringing Model Formats into a ConfigurationSemantic lineage architectureWarehouse dataflow architectureData Flow ArchitectureMetadata Explorer UI3839444949Metadata Explorer UI 50Shapes 50Attachments 52Metadata Search, Browsing and Analysis 545.1Impact of Change Lineage 565.2Data Source Lineage 59Simple Data Governance with Business Glossary 626.1Overview of the Business Glossary features 646.2Creating an abstract Business Glossary 656.2.16.2.26.36.3.16.3.2Create the Business Glossary 65Populate the Business Glossary 66Glossary derived from a Model 72Import the Model 72Create and Populate the glossary 72P a g e 2 220C o p y r i g h t 2 0 1 8 ,e r w i n I n c . a n d / o r i t s a f f i l i a t e s .V e r s i o n 2 0 1 8 R 1 ( 1 0 / 1 6 / 2 0 1 8 )A l lR i g h t sR e s e r v e d .
Metadata Management Tutorial – Web Portal & Data Governance Using erwin MetadataManagement (EMM)6.3.37Update the configuration 76More Metadata Analysis 797.1Terminology Usage (Impact) Analysis 807.1.17.2Trace Semantic Usage 80Trace Semantic Definition 857.2.17.38Build Glossary to Glossary Mappings 87Embed the mapping 94Data Governance with Business Glossary 958.1Glossary Semantic Mapping and Term Classification 968.2Extending the Business Glossary with existing metadata 1068.2.18.2.28.3Analyze potential metadata 106Populate the Business Glossary 108Normalization of Terminology 1118.3.18.3.29Preventing redundancy 111Mapping terminology 1128.4More Editing Terminology in the Metadata Explorer UI 1428.5Report Glossary and Report Documenter 152Data Mapping Specifications 1659.1Constructing a data mapping specification 1669.2Simple Load 1679.3Detailed data mapping specifications 1729.3.19.3.2Include business names 174Creating a map 17510 Advanced Business Glossary Workflow 18110.1Overview of the advanced Business Glossary features nfiguring the Glossary for Workflow 18810.2.110.2.210.2.310.2.410.3Glossary WorkflowWorkflow RolesStewardsWorkflow process optionsTerm management dashboardsWorkflow transition buttonsVersionsCategoriesRelationshipsFlexible and optional workflow processConfigure the GlossaryPublished configuration versionDefining groups to support glossary rolesWorkflow Roles188191193196Glossary Workflow 19910.3.1Propose a draft 199P a g e 3 220C o p y r i g h t 2 0 1 8 ,e r w i n I n c . a n d / o r i t s a f f i l i a t e s .V e r s i o n 2 0 1 8 R 1 ( 1 0 / 1 6 / 2 0 1 8 )A l lR i g h t sR e s e r v e d .
Metadata Management Tutorial – Web Portal & Data Governance Using erwin MetadataManagement (EMM)10.3.210.3.310.3.410.3.5EditReviewApprove a termPublish20120520620711 Semantic Mapping 20911.2Configuration Architecture Diagram 21311.2.1Annotation 213Appendix A 215P a g e 4 220C o p y r i g h t 2 0 1 8 ,e r w i n I n c . a n d / o r i t s a f f i l i a t e s .V e r s i o n 2 0 1 8 R 1 ( 1 0 / 1 6 / 2 0 1 8 )A l lR i g h t sR e s e r v e d .
Metadata Management Tutorial – Web Portal & Data Governance Using erwin MetadataManagement (EMM)1 IntroductionIt is certainly possible to skip through the tutorials, and thus simply glean an“management-level understanding” of erwin Metadata Management (EMM) and its usewithin a metadata management environment. However, it is not recommended that onetry to skip parts of the tutorials and then try to go through later parts. When followingthrough the tutorial sections, it is very important to respect the order of the steps (and theorder sections/labs within each section). The results of preceding tutorials are re-usedand built upon in each successive lesson.In addition, it is important to ensure complete understanding of the conceptualbackground provided in the sections leading up to and supporting the tutorial material.Thus, one should not simply jump into the tutorials with carefully reviewing the conceptspresented in that section.As this document include hand-on tutorials, a great deal of specificity is required. Thisdetail includes specifying particular CASE, ETL, BI, etc., vendor’s tools. While erwinMetadata Management (EMM) environment itself is capable of working with over 100different versions of third-party tools upportedTools.html), it is necessary forthe clarity conciseness of the tutorials to limit the cadre of tools that will be referred to.Please note that it is not necessary to have these tools on-hand to get the full benefit ofthe tutorials. Remember also, though you may intend to use erwin MetadataManagement (EMM) suite of tools with many of the supported third-party tools notspecified in the tutorial, it is still quite valuable to learn the processes, methods and bestpractices presented here. Then one may reuse what one has learned and apply thatknowledge and skill to the particular set of tools that are critical to one’s own enterprise.DisclaimerSome of the features detailed in this document may not apply and/or be available for theparticular erwin Metadata Management (EMM) edition you may have.P a g e 5 220C o p y r i g h t 2 0 1 8 ,e r w i n I n c . a n d / o r i t s a f f i l i a t e s .V e r s i o n 2 0 1 8 R 1 ( 1 0 / 1 6 / 2 0 1 8 )A l lR i g h t sR e s e r v e d .
Metadata Management Tutorial – Web Portal & Data Governance Using erwin MetadataManagement (EMM)1.1 Conventions used in the tutorialThe following font conventions will be used throughout the tutorial. User Interface item – NewSubmenu item – New › FolderTerminology item – model content itemName or label reference – Accounts PayableP a g e 6 220C o p y r i g h t 2 0 1 8 ,e r w i n I n c . a n d / o r i t s a f f i l i a t e s .V e r s i o n 2 0 1 8 R 1 ( 1 0 / 1 6 / 2 0 1 8 )A l lR i g h t sR e s e r v e d .
Metadata Management Tutorial – Web Portal & Data Governance Using erwin MetadataManagement (EMM)1.2 Setting Up the TutorialThis tutorial currently does not contain a separate orientation on the erwin MetadataManagement (EMM) toolset. Specific examples of how to perform the operationsrequired for this tutorial are provided in the next sections.In order to create this structure in the repository for the tutorial, you will restore a backupthat includes the basic repository structure for the tutorial plus any metadata that youcannot easily replicate. To do so1. Sign in as Administrator2. Go to MANAGE Advanced Metadata Manager3. In the Repository panel on the left right-click on the folder named Configuration andselect Delete.4. Now, right-click on the Repository and select More Restore.P a g e 7 220C o p y r i g h t 2 0 1 8 ,e r w i n I n c . a n d / o r i t s a f f i l i a t e s .V e r s i o n 2 0 1 8 R 1 ( 1 0 / 1 6 / 2 0 1 8 )A l lR i g h t sR e s e r v e d .
Metadata Management Tutorial – Web Portal & Data Governance Using erwin MetadataManagement (EMM)5. Click Browse and navigate to the backup at and click More Options Restore.6. Select the following path:C:\Temp\Models \Backups\Data Governance\7. Click Restore8. Refresh the browser.Note how the metadata cataloged in the previous section are reflected in the tutorialrepository structure under the folders Tutorial/Metadata Management.P a g e 8 220C o p y r i g h t 2 0 1 8 ,e r w i n I n c . a n d / o r i t s a f f i l i a t e s .V e r s i o n 2 0 1 8 R 1 ( 1 0 / 1 6 / 2 0 1 8 )A l lR i g h t sR e s e r v e d .
Metadata Management Tutorial – Web Portal & Data Governance Using erwin MetadataManagement (EMM)2About Data Governance and Metadata ManagementFigure 1 -Model Connection Diagram for System Metadata Management ExercisesThe above figure represents an overview of the tutorial that this document will lead onethrough. While it appears here to be a fairly complicated diagram, this document willdecompose and will walk one through the data and metadata management conceptsencompassed.The sample metadata represents what would be captured from operational finance basedsystems, including the operational data stores (ODS), warehouse models (DW andDWStaging), Big Data sources, XML messages and business intelligence (BI) data marts.These will be from a variety of data modeling tools or directly from the sourcesthemselves. In addition, a Business Glossary and enterprise conceptual data model(CDM) are also included and mapped to the operational system metadata.P a g e 9 220C o p y r i g h t 2 0 1 8 ,e r w i n I n c . a n d / o r i t s a f f i l i a t e s .V e r s i o n 2 0 1 8 R 1 ( 1 0 / 1 6 / 2 0 1 8 )A l lR i g h t sR e s e r v e d .
Metadata Management Tutorial – Web Portal & Data Governance Using erwin MetadataManagement (EMM)2.1 General data and metadata management discussionMetadata management is not an end in and of itself. There has to be real return oninvestment in what can be a very expensive and long term undertaking. The promise interms of having a well architected and interoperable information systems environment iswell understood by most people in the information management community. However,as with any foundation, it must be solid, well thought out, and appropriate to the need. Inorder to have an outrageously successful effort, then, the specific objectives must be veryclearly defined, the organization as a whole must be able to leverage the results of themetadata efforts, and the infrastructure must be able to support all of the many methods,design tools, etc., that an organization is using and will adopt into the future.It has been said many times that “Metadata and Architecture assets will outlive the tools,methodologies, and individuals used to produce them.” Thus, it is imperative that yourmetadata be as independent as possible from a particular tool format, methodology orprocess. Having incredibly valuable and expensive work “locked” into a particular tooland methodology or process can spell doom for any metadata and data managementeffort. Metadata must also be accessible to the widest possible audience within anorganization, in order to conclusively demonstrate to the technical and businesscommunities that metadata is a valuable (and accessible) corporate asset.2.1.1 Data and metadata are everywhere!One of the key reasons seen for integrating and managing metadata across an enterprise isthat, as we know, data is everywhere. The proliferation of different software applicationsthat can accomplish more and more specific tasks, and the incredible cost to develop newones, means that inter-operation, or enterprise application integration, with manydisparate data sources is the only way to provide an integrated enterprise. After decadesof trying, we have given up on the concept that there will be just one data source, virtualor otherwise, and interoperability is now known to be the key to integration. If data mustexist in physically separate data stores, on a diverse array of platforms, then integratedenterprise data architecture will be required to “link” the disparate and possibly redundantdata stores together via well-defined processes. Losing track of these “links” costsorganizations millions of dollars annually, by increasing the risk of system errors,extending application development project timelines, increasing the cost/complexity ofbuilding decision support systems, and countless other ways. The inability to traceinformation across disparate processes and technologies has lead to a lack of trust bybusiness management about the reliability of the content of corporate information.Having to then assure the quality of the information used to make the decision, inevitablycauses delays in key business decision making. Thus is born the need for metadatarepositories, metadata integration and management.P a g e 10 220C o p y r i g h t 2 0 1 8 ,e r w i n I n c . a n d / o r i t s a f f i l i a t e s .V e r s i o n 2 0 1 8 R 1 ( 1 0 / 1 6 / 2 0 1 8 )A l lR i g h t sR e s e r v e d .
Metadata Management Tutorial – Web Portal & Data Governance Using erwin MetadataManagement (EMM)What needs to be recognized, though, is that metadata is also everywhere. DataModeling tools abound in an organization, many with their own repositories, just aboutevery application has its own data dictionary, XML carries the metadata with it in themessage or document, and data flow mapping inter-relate all of it together. Thus, thesame difficult lesson applies: the truly effective and future proof way to integrate andmanage this metadata is interoperability and enterprise repository integration. There isnever going a single repository (metadata source), virtual or otherwise, and thus,metadata exchange and repository interoperability are key to metadata integration.2.1.2 MITI’s strategy to support data and metadata managementIn order to succeed, then, one must have a good enterprise repository integrationenvironment. This philosophy is what the erwin Metadata Management (EMM)repository solution and products are based upon. It is an integrated whole that can bridgethe technical and non-technical aspects of metadata, while simultaneously bridging thechasm between the different metadata source and target systems that constitute anymodern information management environment.Metadata is everywhere and erwin Metadata Management (EMM) provides datagovernance metadata repository environment that is agnostic of data modeling toolsource or other data store metadata sources. Including the business glossary, data flowmapping design, semantic mapper and bridges into big data, database technologies,P a g e 11 220C o p y r i g h t 2 0 1 8 ,e r w i n I n c . a n d / o r i t s a f f i l i a t e s .V e r s i o n 2 0 1 8 R 1 ( 1 0 / 1 6 / 2 0 1 8 )A l lR i g h t sR e s e r v e d .
Metadata Management Tutorial – Web Portal & Data Governance Using erwin MetadataManagement (EMM)XML, COBOL, etc., erwin Metadata Management (EMM) allows one to manage andanalyze the complete enterprise metadata life cycle across heterogeneous vendors.P a g e 12 220C o p y r i g h t 2 0 1 8 ,e r w i n I n c . a n d / o r i t s a f f i l i a t e s .V e r s i o n 2 0 1 8 R 1 ( 1 0 / 1 6 / 2 0 1 8 )A l lR i g h t sR e s e r v e d .
Metadata Management Tutorial – Web Portal & Data Governance Using erwin MetadataManagement (EMM)3 Metadata HarvestingIn order to design, build and maintain a metadata management environment, one mustidentify and catalog the assets in the current environment. In general, this process isbroken into the cataloging of data stores, data processes and conceptual models.P a g e 13 220C o p y r i g h t 2 0 1 8 ,e r w i n I n c . a n d / o r i t s a f f i l i a t e s .V e r s i o n 2 0 1 8 R 1 ( 1 0 / 1 6 / 2 0 1 8 )A l lR i g h t sR e s e r v e d .
Metadata Management Tutorial – Web Portal & Data Governance Using erwin MetadataManagement (EMM)3.1From Data Stores and Data Processes To Enterprise Architecture3.1.1 Identify data stores & associated design toolsWith that overview of the erwin Metadata Management (EMM), we may now return tothe general metadata management discussion. One of the first steps in populating therepository and thus building up one’s metadata management environment is an inventoryof the metadata in your organization.For the purposes of this document, the specific methodology used is immaterial. Instead,from the MIR perspective it is most important to analyze where the metadata resideswhat technology is required to extract it (and ultimately forward-engineered)what process or set of steps must be followed in order to ensure proper extractionAs was stated in previous sections, metadata is everywhere within any moderninformation management organization. Also, there are many repositories that arerequired to exist for each technology environment in order for it to perform properly. Inorder for these sources and targets to provide a coherent and coordinated environment,this metadata must be exchanged.3.1.1.1 Interface technology issuesFundamental to the repository is the ability to
Metadata Management Tutorial – Web Portal & Data Governance Using erwin Metadata Management (EMM) P a g e 9 220 2 . Metadata Management Tutorial – Web Portal & Data Governance Using erwin Metadata Management (EMM) Metadata Management Tutorial – Web Portal & Data Governance Using erw
