WIXLIB

An Oracle White PaperAugust 18th, 2017Oracle Metadata Management v12.2.1.2.0New Features Overview

Oracle Metadata Management version 12.2.1.2.0 – August 18th, 2017New Features OverviewDisclaimerThis document is for informational purposes. It is not a commitment to deliver any material, code, orfunctionality, and should not be relied upon in making purchasing decisions. The development, release,and timing of any features or functionality described in this document remains at the sole discretion ofOracle.This document in any form, software or printed matter, contains proprietary information that is theexclusive property of Oracle. This document and information contained herein may not be disclosed,copied, reproduced, or distributed to anyone outside Oracle without prior written consent of Oracle.This document is not part of your license agreement nor can it be incorporated into any contractualagreement with Oracle or its subsidiaries or affiliates.1

Oracle Metadata Management version 12.2.1.2.0 – August 18th, 2017New Features OverviewTable of ContentsExecutive Overview .3Oracle Metadata Management 12.2.1.2.0 .4HARVESTED MODEL CHANGE DETECTION .4SUBSCRIPTION / NOTIFICATION.4CUSTOM ATTRIBUTES.4BUSINESS GLOSSARY WORKFLOW .5DATA INTEGRATION BROWSING AND LINEAGE .5DATA CONNECTION STITCHING MAJOR IMPROVEMENTS .6BULK DATA MAPPING . 6LIGHTWEIGHT MODELS . 6MODEL COMPARE/MERGE .6DATA MODELING.7REST API SDK.7Metadata Harvesting (Model Bridges) Improvements .7METADATA HARVESTING FROM CLOUD DEPLOYED MM/DG SERVERSDATA STORES (Flat files, RDBMS, Cloud Databases, Big Data / Hadoop):DATA INTEGRATION (DI/ETL/ELT):.9BUSINESS INTELLIGENCE (BI/OLAP):.9DATA MODELING (DM) and Object Modeling:.10METADATA MANAGEMENT (MM) & DATA GOVERNANCE (DG):10782

Oracle Metadata Management version 12.2.1.2.0 – August 18th, 2017New Features OverviewExecutive OverviewThe Oracle Metadata Management (OMM) solutions include two products: the Oracle Metadata Management for Oracle Business Intelligence (OMM4OBI) and the Oracle Enterprise Metadata Management (OEMM)Oracle Metadata Management for Oracle Business Intelligence is a software package for metadatamanagement of Oracle environments. Oracle Metadata Management for Oracle Business Intelligenceincludes the following metadata management features: Metadata Harvesting from Oracle technologies Metadata Configuration and Stitching Metadata Browsing, Search and Reporting Metadata Collaboration (external URL, tagging, comments and review) Data Flow Lineage & Impact Analysis Metadata Explorer (simplified metadata user interface for business users)Oracle Enterprise Metadata Management is a software package for metadata management of multivendor environments along with support for data governance. Oracle Enterprise MetadataManagement includes all features of Oracle Metadata Management for Oracle Business Intelligencewith the following extra metadata management features: Metadata Harvesting from multi-vendor technologies Metadata Version and Configuration Management (change management) Data Model Diagram Visualizer and Navigator Business Glossary for Data Governance Semantic Lineage & Impact Analysis Semantic Mapping Editor Data Flow Mapping Specifications Editor Data DocumenterThis whitepaper describes in detail some of the new features and capabilities offered in OracleMetadata Management with the release of version 12.2.1.2.0.3

Oracle Metadata Management version 12.2.1.2.0 – August 18th, 2017New Features OverviewOracle Metadata Management 12.2.1.2.0HARVESTED MODEL CHANGE DETECTIONThis feature avoids systematically creating a new version of the content when schedulingthe harvest of large databases, data models, data integration jobs, or business intelligenceservers. Change detection is automatically performed during any new harvesting by asystematic and efficient (fast) comparison with the previously harvested metadata. Thisability dramatically reduces the needed database space by creating fewer versions, andenables the possibility of reliable notification when a change really occurred. Note that inprevious versions, the incremental harvesting already offered systematic and automaticreuse of unchanged models (e.g. BI Reports) from a large a multi-model server (e.g. BIserver).SUBSCRIPTION / NOTIFICATIONOMM has added a new mechanism to operate in response to changes at both therepository model level (e.g. a new version of a data store is harvested) and at the modelobject level (e.g. a terms changed or ready to approve/review in a Business Glossaryunder Workflow). Anyone may be assigned stewardship roles and thus will be notified asnew imported content is harvested, with links back to the new object and the ability tocompare using the newly re-written powerful comparator, and the ability to identifyimpacts of change for any architecture or configuration of assets.CUSTOM ATTRIBUTESCustom attributes may now be applied on any harvested metadata (models) from datastores, data modeling, data integration, and business intelligence tools. Thus allowing totag any metadata (from Hive tables to BI reports down at the column/field level) forcustom properties such as a company confidential level that can be read for externalproducts on security enforcement and compliance. Note that in previous versions, thesesame custom attributes could already be applied to authored metadata (models) such asterms of Business Glossaries or tables/columns of Data Models.4

Oracle Metadata Management version 12.2.1.2.0 – August 18th, 2017New Features OverviewBUSINESS GLOSSARY WORKFLOWEven with workflow enabled, business glossary terminology may now be edited by thebusiness users behind the Metadata Explorer UI. Note that the in previous version, a(customizable) Workflow was already available in the Metadata Manager.DATA INTEGRATION BROWSING AND LINEAGEAllowing business users behind the Metadata Explorer UI to browse data integration jobs(from DI/ETL/ELT tools as well as SQL Scripts) and analyze summary data flow lineageof their execution. Note that in previous versions, the browsing (open) and full detaileddata flow lineage of any data integration models (DI/ETL/ELT and SQL scripts jobs)were already available in the Metadata Manager (and still are).5

Oracle Metadata Management version 12.2.1.2.0 – August 18th, 2017New Features OverviewDATA CONNECTION STITCHING MAJOR IMPROVEMENTSOMM is now providing the ability to stitch by column position (needed in SQL insertstatements of ETL scripts ), and smart case aware stitching (as some database's namespaces like schema/table/column are case sensitive while others are not).BULK DATA MAPPINGData mapping specifications may now be defined as bulk mappings for all columns in atable.LIGHTWEIGHT MODELSModel content (such as a harvested databases or data models) can be stored in therepository as a lightweight model (just the XML file), or fully expanded (as both XML fileand fine grained repository objects). When retaining many historical versions of a model,using lightweight models saves repository space, and also avoids slowing down the searchby not indexing historical repository objects. Lightweight models cannot be directly usedin a Configuration or in a Mapping. However, the lightweight model of a data store (suchas a RDBMS or Hadoop Hive) can be documented with a reference to its associatedPhysical Data Model (PDM) for data model diagramming and semantic linking to aBusiness Glossary (BG). Such a PDM can of course be used in any Mapping orConfiguration to be exposed to business users in the Metadata Explorer. Note thatlightweight models can immediately (without any loss of performance) be opened in theMetadata Manager (to browse metadata or trace lineage within that model), Compared(with the Model Comparator to analyze the difference between versions), Exported (forexample to BI design tool).MODEL COMPARE/MERGEThe comparison facility has been completely re-written to include comparison for everylevel of detail for those models with the same profile (e.g., data model from onetechnology and data model from another). Even entirely different contents (e.g., an ETLand a data model, or a BI Design and Glossary) may be compared, only at a lessor level ofgranularity (basically at the granularity of stitching, e.g., schema, table, column). Finally,for physical data models (including documentable models based upon harvested databasestructures) one may use a powerful merge feature again with full control down to any levelof granularity6

Oracle Metadata Management version 12.2.1.2.0 – August 18th, 2017New Features OverviewDATA MODELINGMajor improvements and new features involving the physical data modeling capabilitiescentered around a data documentation tool enabled through physical data modeling basedupon the structure harvested from existing data stores of the of data warehouse and datalake (including traditional RDBMS and Hadoop Hive big data). Also includes newadvanced support for physical data model diagram editing, relationships definition, all theway to export to data modeling tools, business intelligence design, etc., andcomparison/compliance with live database / big data hive implementations.REST API SDKmajor enhancements with many new features.Metadata Harvesting (Model Bridges) ImprovementsMETADATA HARVESTING FROM CLOUD DEPLOYED MM/DG SERVERSTwo possible deployment architectures for the MIMB server:7

Oracle Metadata Management version 12.2.1.2.0 – August 18th, 2017New Features Overview Local Network: For a Model Bridge Server connected to a local MetadataManagement Server via a dedicated open port on the same network where firewallis not an issue. In such case, the Model Bridge server is controlled by the MetadataManagement Server in a classic client/server architecture.Over the Web (Cloud): For a Model Bridge Server connected to a remoteMetadata Management Server over the Web where a secure firewall is in place. Insuch case, the Model Bridge Server will proactively connect over http on regularbasis to a dedicated remote Metadata Management Server for any metadataimport/export activities to perform. This is the recommended architecture forMetadata Management Servers hosted on the cloud needing to access local ModelBridge Servers behind a firewall for metadata harvesting purpose.DATA STORES (Flat files, RDBMS, Cloud Databases, Big Data / Hadoop):oDATA CATALOGING OF THE DATA LAKE over the cloud (e.g. Amazon S3), onbig data clusters (e.g. Hadoop distributions), or file servers (POSIX file systems)- with high level data lake directory inventory import bridges such as: Amazon S3 (Simple Storage Service) import (beta bridge) Apache Hadoop Distributed File System (HDFS Java API) import or(WebHDFS REST API) import (beta bridge) Posix (Windows/Linux) File System import (beta bridge)- discovering lower level metadata from the database and file import bridgessuch as:oo Flat File Database (CSV or Excel) import (beta bridge) JSON (JavaScript Object Notation) Database Files import Apache Avro Database Files (JSON) import Apache Parquet Database Files importBIG DATA HADOOP HIVE TECHNOLOGIES (Cloudera, Hortonworks, MapR) Apache Hadoop Hive Database (HCatalog via JDBC) import Apache Hadoop Hive Database (Web Hcatalog) import Apache Hadoop Hive Database (Hive Metastore via JDBC) import (betabridge)NO SQL TECHNOLOGIES MongoDB Database (JSON) import Apache CouchDb Database (JSON) import MarkLogic NoSQL Database (for JSON only) import Apache Cassandra Database import DataStax Enterprise (Cassandra) import8