Transcription
Metadata and the Rise of Big DataGovernance: Active Open SourceInitiativesOctober 23, 2018
Today’s speakersJohn Mertic,Director of ProgramManagement, LinuxFoundationDavid Radley, ODPiEgeria maintainer,IBM2@ODPiOrg
Today’s reality@ODPiOrg
Imagine An enterprise data catalog that lists all of your data,where it is located, its origin (lineage), owner,structure, meaning, classification and qualitySpanning systems both on premises and within cloud providersHosted locally to your data platforms but integrated to providethe enterprise view@ODPiOrgSearch
Imagine SearchNew data tools (from any vendor) connectto your data catalog out of the boxNo vendor lock-in; nor expensive population of “yet another” proprietary siloed metadatarepository@ODPiOrg
Imagine Metadata is added automatically to the catalog as new data is createdExtensible discovery processes characterise and classify the dataInterested parties and processes are FunctionFunctions
Imagine Subject matter experts collaboratingaround the dataLocating the data they need, quickly and efficientlyFeeding back their knowledge about the data and theuses they have made about it to help others andsupport economic evaluation of data@ODPiOrg
Imagine Automated governance processes protectand manage your dataMetadata-driven access controlAuditing, metering and monitoringQuality control and exception managementRights management@ODPiOrgDatabases
A new manifesto for metadata and governance!Metadata management must be automatedMetadata management must become ubiquitousMetadata must become open and remotely accessibleMetadata should be used to drive the governance of dataThe discovery, maintenance and use of metadata has to be an integral part of alltools that access, change and move information.@ODPiOrg
How will this be achieved?Open andUnified Metadata@ODPiOrg
Open metadata management ecosystemPeer-to-peer network of repositoriesCollaborationSpace MetadataMetadata stored and managed closeto its sourceEach repository/tool bringsunique value.Analytics PlatformMetadataOpen, extensible metadata structures formetadata exchange and federationCloud SaaS platformMetadataApplicationMetadataHadoop PlatformMetadataOpen source infrastructure sharing cost of development and maintenance@ODPiOrg
Making Metadata Available to the Enterprise!Data LakeMarketingData LakeCohort ACohort BChief Data OfficeSystems of RecordMobileApps@ODPiOrgSystems ofRecord12
Open metadata data modelGlossaryCollaborationGovernanceModels andReference DataLineageBase Types, Systemsand Infrastructure@ODPiOrgData AssetsMetadataDiscovery
Open metadata and governanceintegration patternsApacheAtlas@ODPiOrgIBM InformationGovernance Catalog
Business meaning of the underlying CORDHAS-AHAS-AHAS-AEmployee IdEmployee NameEMPNAMEEMPNOJob TitleSensitiveWork LocationCompensation PlanIS-AIS-AHourly Pay RateAnnual ata fora data storeData00 3809890 6 7 Lemmie Stage 818928 3082 4 New York 4 27 DataStage Expert 1 45324 300 27 Code St Harlem NY 1 316
Instance representations in Open Metadata Resource AttributesPrimitivesEnumsCollections17
Open Source Collaboration through ODPiODPi is the vendor neutral home for Egeria.Governance is open to all in the data governance communityVendorsEnd-usersPrivacy/governance expertsCode is available under an Apache 2.0 license and documentation under a CCBY-4.0 licenseJoin the effort at https://github.com/odpi/egeria@ODPiOrg
How this helpsData Governance ProfessionalsVendorsYour governance program if based on establisheddefinitionsYour metadata offerings will deliver value faster asthey tap into metadata collected by other vendor’stools.Allow a broader range of tools in your organizationAutomated governance processes protect andmanage your dataMetadata-driven access controlODPi packages extend your metadata system’sand tools’ capabilitiesConformance tests minimize your effort in beingcompliant with key standards and regulations.Auditing, metering and monitoringQuality control and exception managementRights management@ODPiOrgCustomers have increased confidence in yourtools and services due to ODPi certification.
ODPi – A neutral home for collaboration@ODPiOrg
Look to The Linux FoundationThankfully, that’s where The Linux Foundation comes in. For nearly two decades, The LinuxFoundation has provided unparalleled support for open source communities through financial andintellectual resources, governance structure, IT infrastructure, services, events, and training.Dedicated to building sustainable ecosystems around open source projects, The Linux Foundation is working with the globaltechnology community to solve the world’s hardest problems through open source and creating the largest shared technologyinvestment in history.The Linux Foundation is the umbrella organization for more than 60 open source projects accelerating open technology developmentand commercial adoption. Some of the game-changing initiatives hosted by The Linux Foundation include:@ODPiOrg
Get involved!Check out ODPi Data Governance on GitHubhttps://github.com/odpi/data-governanceLearn more about Egeriahttps://odpi.github.io/egeria/Follow the vernanceHave your organization support ODPihttps://lists.odpi.org/about/join@ODPiOrg
zzzzzzzQuestions?@ODPiOrg
@ODPiOrg
Your governance program if based on established definitions Allow a broader range of tools in your organization Automated governance processes protect and manage your data Metadata-driven access control Auditing, metering and monitoring Quality control and exception management Rights management Your metadata offerings will deliver value faster as
