WIXLIB

Metadata Management Best Practicesand Lessons LearnedPresentation at2006 DAMA / Wilshire Metadata ConferenceDenver, COJohn R. Friedrich, II, PhDFriedrich@metaintegration.netMetadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 1 of ?

Outline Recent developments in metadatamanagement New opportunities New challenges and Lessons Learned ConclusionMetadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 2 of ?

Format of This Presentation Outline to “stay on the path”Background to “level the playing field”Example for clarity of understandingReal-time example for credibilityMetadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 3 of ?

Recent Developments in MetadataManagementWhat is “new” out there?Metadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 4 of ?

Recent Developments: Metadata ExchangeSupported by Vendors Nearly all recognize the need for metadataexchange– Especially across different “types” of tools Warehouse design to ETL or BI ETL to lineage analysis tool BI to Enterprise Reference Model E.g., Multi-Vendor panel with 14 panelist– Each one has metadata exchange capabilities– Most built in to the toolsMetadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 5 of ?

Recent Developments: Multi-vendor MetadataAccessibility Metadata hubs with multi-vendor capabilities inone product– Over 90 products integrated into a tool– “Metadata services” Not just “one stop shopping” for metadata, but formetadata accessibility servicesMetadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 6 of ?

Recent Developments: Automated and EfficientMetadata Access Not just services, but automation services– Server based– Process based– CustomizableMetadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 7 of ?

New OpportunitiesOut of these developments comeopportunities.Metadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 8 of ?

New Opportunities: Multi-vendor MetadataAnalysis Accessibility Metadata Storage Î Throughout the entire data lifecycle– Operational Data Stores– ERP– ETL– EAI– EII– DW– BIRepositoryMeta-DataAnalysisMeta-Data HubETL ryDW BOUniverseER/StudioPowerDesignerCOBOLThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumBODesignerCrystalReportsDW SchemaModelMartBusinessObjectsInformatica tadata Management Best Practicesand Lessons ETLCognos sReportsReportsReportsReportsMetaStageApr 23-27, 2006Denver, COSlide 9 of ?

New Opportunities: Multi-Vendor orySystemArchitectMeta-Data HubETL ryDW BOUniverseDW SER/StudioCOBOLBusinessObjectsInformatica LCognosFrameworkManagerCognos rtsReportsReportsDataStageMetadata Management Best PracticesThe 10th Annual Wilshire Meta-Data ConferenceMetaStageand Lessons Learnedand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 10 of ?

Show and TellLet us stop and build somethinghere.Metadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 11 of ?

New Opportunities: Up-To-Date Physical (andLogical) Metadata Accessibility Automation Î The “pull”– “As close to the grove as you can get” physical metadata– Physical (real-world or data tool) driven data life-cycle ETL transforms really can define the data flow in the repository– Logical lineage derived from physical “reality” The “push”ODS– LogicalmetadataETLin tools reflects architecture workODSODSReports– Physical ODSmetadatareuse and changepropagationReportsODS The processODSETLDataWarehouseReportsReports– Good metadata management and lifecycle processReportsReportsautomationMetadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 12 of ?

New Opportunities: What-If Impact Analysis Accessibility Automation Process Ζ Not just “one version of the truth”– Multiple future “configurations” of metadata may becaptured– Analysis of change impacts upon all of these to beor proposed configurations– Deployment planning– Impact risk assessmentsMetadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 13 of ?

New Opportunities: Historical Business-OrientedLineage Analysis Accessibility Automation Time Ζ Reverse lineage (“where did it come from”) is oftenan historical question– Sarbanes-Oaxley is for a year, at least– BASEL II is up to five years of history– Last quarter’s sales is last quarter– Today’s “version of the truth” is not yesterday’s, justas it is not tomorrow’s (what if impacts)Metadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 14 of ?

New ChallengesIf it can be done, it has been, inone form or another.Only the unlikely or impossible areworth striving for.*Metadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 15 of ?

New Challenges: Multiple RepositoriesDevelopmentMetadataRepositoriesThe development and operational metadata repositoriescan be the same product (development vs. production instance)or the operational repository can be a specific productwith only run time metadataData Modeling Toole.g. CA AllFusion ERwinMetadata import/ exportModelManagerAnalysisMetadataRepositoryThe life cycle and analysis metadata repositoriescan be the same ionalETLETL Development ToolPowerCenterLife riesVersion & configurationManagementMetadata ComparisonMetadata IntegrationMetadata MappingMetadata DW / BIMetadataone-wayMetadata StitchingETLe.g. InformaticaMetadata Lineage& Impact AnalysisDevelopment to productionMetadata ReportingRun-time(execution log)MetadataBI Development ToolFrameworkManagere.g. Cognos ReportNetDevelopment to productionMetadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 16 of ?

Lessons Learned: Multiple Repositories Learn from the data lessons– A single grand repository, like a single grand database, isnot going to happen “Embrace diversity”: Use the ETL tool to describe data movement transformations andworkflows, the BI tool for Cubes and reports, the CASE tool fordesign, etc. Pitfalls of the “round-trip” Capture tool-specific metadata, share normalized metadata. Remember the word “standards” always has an “s” onthe end of it!Metadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 17 of ?

New Challenges: Version Management Many repositories and tools x many models xtime and change Ζ A version for each!– Several new dimensions to the repository– Answer the difficult questions, not the “singleversion of the truth” assumption-based onesMetadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 18 of ?

Lessons Learned: Version Management Need true version management– Maintain multiple versions, not just deltas– Historical path (version traceability)– Process (milestone) driven– Fully automated (don’t muck around in therepository) Bonus: Process based metadata qualityMetadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 19 of ?

New Challenges: Configuration Management Versions x deployments x what-ifs x organizationalstructure x . . . Ζ True configuration management with many configurations ofmany versions– Many dimensions of CM problem: Multiple deployed versions of each of the source systems,Multiple design, developmental, beta, etc.Multiple version of standards and/or reference modelsMultiple versions of data migration transformationsMultiple business organizational “cuts”Multiple IT organizational “cuts”And many, many moreMetadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 20 of ?

Lessons Learned: Configuration Management There are many ways to slice it Must plan ahead Tie configuration organization to:––––Data Flow!IT deployment an responsibilitiesMilestonesBusiness organization Manage fundamental (separately versioning)components separately in the data flow Most of your time will be spent telling the metadatawhat the separate tools did not understand about eachother Î STITCHINGMetadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 21 of ?

New Challenges: Automation, Processes andMetadata Quality Complexity of access processes, versions, andconfigurations Ζ Must automate– Must automate metadata management (which aredata management driven) processes– Automation means making mistakes very quickly,so must ensure quality of metadata, version andconfigurations– Don’t want to go to jail due to a bad SOX answer!Metadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 22 of ?

Lessons Learned: Automation, Processes andMetadata Quality This is meta-automation (I guess) Repository (metadata) administration is NOTvery often administration of the repository(metadata) Repository is most often administration of theprocesses These processes must be derived from the dataprocesses As with SOX, quality comes implicitly from, andis monitored by way of the processMetadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 23 of ?

ConclusionMetadata Management Best Practicesand Lessons LearnedThe 10th Annual Wilshire Meta-Data Conferenceand the 18th Annual DAMA International SymposiumApr 23-27, 2006Denver, COSlide 24 of ?