WIXLIB

IBM Replication – Data DeliveryThe DataView Show – June 2020A Premier IBM Business Partner

Agenda 2IntroductionReplication Review Why do we replicate data? Competitive LandscapeIBM Replication ProductsIBM Replication Components

Agenda (contd.) 3Sources Classic mainframe: Db2 for z/OS, IMS,VSAM Remote Captures HeterogeneousTargets Native targets Federated targets Message queues Event Publishing Kafka

Agenda (contd.) 4Key featuresUpcoming enhancements Remote Capture for VSAM Integrated SynchronizationToolingDecision criteriaPurchasingResources

IntroductionThe Fillmore Group, Inc. Founded in the US in Maryland, 1987IBM Business Partner since 1989Delivering IBM authorized education since 1994IBM Gold Consultant since ants 5IBM Champions since 2009

Poll 1: Is your organizationcurrently using replication in aproduction environment?6

Poll 2: Which replication solutionsare in use at your organization?7

Replication in Action - Whythat ALL of your business is driven Major z/OSEnsurerevenue driver Give it away for Db2up-to-the-secondon other platformsaccurate databy Business in Real Time-Detect and react to data events as theyhappen to drive the business-Optimize decision making with up tothe second data, i.e. real time analytics Always On Information- High availability with Active-Standby andActive-Active data deployments- Data synchronization for zero downtime data migrations and upgrades Major z/OSrevenueProvidethedriverdata synchronization underpinnings High value embedforforcontinuousDb2 on otherplatformsdataavailability8

Keeping data synchronized among applications is and will bethe most prevalent use case for data integrationPlease indicate what use cases of data integration and integrity solutions have beenimplemented, being implemented or are planned for in your environment.9

Replication - Core Values(More Why)Optimize resource utilizationSend only the changes with continuous“feeds”. Read Source only for "refresh”Extend Application AvailabilityShorten batch windows by streamingchanges in real time as they occurImprove the “Bottom Line”Fresh data improves business resultsReduced network traffic/costsAudit capabilities for trusted dataEnterprise readyTransactional integrity with very highvolume throughput and low eScale

Poll 3: What are the purposes ofreplication in your organization?11

Gartner MagicQuadrant for DataIntegration ToolsAugust 2019ID G00369547https://www.gartner.com/doc/reprints?id 11OJ15E39&ct 190923&st sb12

IBM Strengths - Gartner “Depth of integration offering. Reference customers highlightedthe completeness of IBM’s holistic data integration suite, including itsrich functionality, variety of prebuilt functions and connectors, and itsoverall performance.” “Diverse data integration delivery styles. Reference customersuse IBM’s products for traditional data delivery styles (datareplication, batch processing), as well as more complex data deliverystyles (including data synchronization and stream data integration).They praised IBM’s data integration tool portfolio for its ability todeliver complex data integration requirements that demandcombinations of traditional and modern data integration styles, suchas data replication, data virtualization and stream data integration forreal-time analytics.”13

IBM Strengths – Gartner (contd.) 14“Brand awareness and market presence. IBM’s size and theglobal coverage of its business systems, infrastructure platforms andanalytics solutions enable it to draw on a huge customer base and awide product distribution model for positioning its data integrationtools. Broad usage of IBM technologies within its customer base hasdriven the wide availability of implementation service providers andapproaches to solving complex integration challenges.”

IBM's Replication Portfolio - WhatDynamically synchronize changing structured data with diverse targets, apps, platforms Real time updating using continuous delivery with full refreshLow impact on source applications using log files for continuous, not source dataDatabasesData from any source to any target with transactional integrity- ODS, HA copy, DB migration,cross application integrationProven enterprise-class performanceDataStageAudit DatabaseInfo. ServerGUIsfor Admin& Monitoringor other ge queueBig hive)CaptureApplyKafka

IBM Replication Products 16IBM Data Replication (IDR)IBM InfoSphere Data Replication (IIDR)Db2 for z/OS Remote CaptureIBM InfoSphere Classic Replication (for VSAM, IMS, andwith Federation to others)IBM Data Replication for Db2 Continuous Availability

IBM Replication Components 17SQL Replication Staging Tables Used in broadcast topologyQ Replication IBM MQ High volume, low latencyChange Data Capture (CDC) TCP/IP Heterogeneous

Supported source databasesIBM Db2 for Linux, UNIX and Windows (LUW)IBM Db2 for iIBM Db2 for z/OSIMSMicrosoft SQL ServerOracleSybaseInformixDb2 on CloudVSAMPostgreSQLDb2 Warehouse on CloudDb2 WarehouseMySQL18Supported target databases andmiddleware applicationsIBM Db2 for Linux, UNIX and Windows (LUW)IBM Db2 for iIBM Db2 for z/OSIBM InfoSphere DataStageIBM NetezzaIBM InformixMicrosoft SQL ServerMicrosoft Azure SQL DatabaseMicrosoft Azure SQL Database Managed InstanceOracleSybaseCDC Replication Engine for Event ServerCDC Replication Engine for FlexRep (JDBC)IBM CloudantIBM Integrated Analytics SystemApache HadoopApache KafkaDb2 Warehouse on CloudDb2 on CloudDb2 WarehouseIBM MQ for z/OS (using Classic CDC for z/OS)Teradata

Always On InformationBusiness in Real TimeDB2DB2IMSDB2 (z/OS, i, LUW)Oracle/ExadataMS SQL ServerInformix & SybaseIMSVSAMVSAMStrategic ObjectiveDb2 WHDb2 WHIIASIIASIIASDb2 WH19IMSVSAMDB2 (z/OS, i, LUW)Oracle/ExadataMS SQL ServerInformix & SybaseBig Data – Kafka and HadoopAppliances – IIAS, IPSDb2 WarehouseTeradataMySQL, EnterpriseDBAzure SQL DatabaseCloud (IBM/AWS/Azure) using BYOLInformation Server (DataStage)FilesJDBC TargetsODBC TargetsQueues,

IBM’s Replication Portfolio – Technologies with Use CasesBusiness in real timeAlways on sageHTAPARep4CAHTAPBData LakeCDC direct apply for real time analytics, Stand-alone replicationapplication integration:and integration withGDPS Active/Active: Hadoop (WebHDFS) Cloud & On-Premise Cloudant Db2 Warehouse IIAS Kafka DataStage General Relational DB Targets All Db2, MS SQL,Oracle/Exadata, CDC and Classic CDC Any Db2 to/from Any Db2 IMS to IMS VSAM to VSAMQ-Rep / IMS Rep / VSAM RepContainerized replicationtightly integrated with Db2Warehouse and IIAS:1. IIAS to/from IIAS2. IIAS to/from Db2 Warehouse3. Db2 Warehouse to/from Db2WarehouseReplication forContinuous Availability

Key Features21

Poll 4: Do you replicate data fromIBM z Systems?22

Remote Capture from Db2 z/OS – Business ValueIBM Data Replication for Db2 z/OS Remote Source Share mission critical Db2 z/OS data with new environments for Analytics, integrationand cloud projects including targeting:– RDBMS data warehouse– Kafka hub– Information Server (ETL solutions) ̶ OLAP appliances (IIAS) and databases̶ HDFS clusters (Hadoop)̶ Cloud targetsDeploy the replication capture remotely from the mainframe 23Reduce z/OS MIPS needed to replicate Db2 for z/OS data by up to 50%Reduce dependency on specialized System z skills for:– Deploying: installation from LUW platform– Configuring: via Management Console UI or scripting– Monitoring: via Management Console UI or scripting

Remote Capture from Db2 z/OS – Deployment and OperationsLUW*Z/OSDb2 for z/OSv11 or v12Non-z CDCTargetPlatform**IDR/IIDRStoredProcedureDb2 z/OS Log Recordsusing JDBCActive orArchiveLogsDb2 z/OS RemoteSource Process IDR/IIDRCDC CaptureEngineCDCTargetEngineTCP/IPWLM EnvironmentRuntime EnvironmentExternal Stored Proc.APF Load LibraryStored ProcedureLoad Module(uses IFI interface)Initial DeploymentProcessWorking LibraryInstance CreationDeploys native codeusing an SSH port*“Linux-Unix-Windows” Source platforms per IDR support matrix** Target platforms vary by the target itself and are also in the IDR supportmatrixNOTE: Target Engines can be installed on a Source platform if target is supportedthere

Hadoop HDFS and KafkaHadoop HDFSKafka Designed to process large relativelystatic data sets Designed from the outset to deal with constantlychanging events/data Bulk append only, no update resulting in manyfiles, more storage Built in Insert plus log compaction (Delete)to emulate an update HDFS file system designed to distribute copies ofdata across commodity nodes to provideavailability & scalability at low cost Also exploits commodity nodes for scalability,availability and cost but with self managed storagedramatically reduced footprint No set format to the data, Source/Consumeragree on record layouts in advance Storage is self described JSON (Java Script ObjectNotation) document wrapped in Apache Avrobinary format25

Replication’s Real Time Data Feeds to Kafka Two Write methods: REST API well suited to targeting managed environments, restricted access Native java API higher performing for inside the firewall or with "tunnels"Some Details: Targeted Kafka must be at 0.10 standard level (a.k.a. Kafka 10) or higher We recommend a schema registry service and corresponding deserializer that supports the Confluent openAPIsOR Use provided Kafka Custom Operator Process (KCOP) that eliminates the use of a schema registryCaptureLog26PushApply

What are CDC Kafka customers doing? Architectural Data Hub / Landing Zone Staging for Data Lake and other Analytics platforms Handles OLTP transactions, staging results for consumption by non-OLTPtargets Hadoop/HIVE, Column Store Databases, Key Value Databases (Aggregation, Net Change)Staging for the Cloud The central point in the architecture for all data to be landed and consumedby various appsData is replicated to Kafka, in some cases manipulated (Compaction, StreamsJobs), Results / original data then copied to the CloudDelivering OLTP data directly to Cloud 27Data written to Cloud-hosted Kafka for enterprise use

PostgreSQL Capture Engine PostgreSQL is the #1 growth database in today’s marketShare PostgreSQL data into new environments forAnalytics and Integration Delivered via continuous delivery (fixpack) into IDR/IIDRv11.4 28

Replication for Continuous Availability IIAS - IIASDb2WH - Db2WH and Db2WH - IIASProvides asynchronous transactionally consistent Db2replicationNear real-time mirror of primary platform data onsecondaryBoth primary and secondary are active29

Upcoming Enhancements*Remote Capture for VSAM – June 2020 MIPS offload Requires VSAM logging Early availability/beta Integrated ore 30

Tooling Out Replication Center Q Replication DashboardIn Watson Knowledge Catalog Watson Studio Enterprise Db2 Unified Console31

User Interface – integrated into IIAS console32

Poll 5: Which of the IBM replicationsolutions presented sounds *most*interesting based on yourorganization’s needs?33

Decision criteria 34Cost Software: licensing, maintenance Implementation BAU processing cyclesData volumeTolerance for data lossLatencySkillsSimplicity vs. complexityRisk

How to BuyStep 1: Identify all sources and targets Step 2: Determine whether replication will beunidirectional or bidirectional Step 3: Determine the best IBM replication solution Step 4: Figure out how to get YOUR best deal 35

IBM Products and MetricsProcessor Value Units (PVU) – used for all nonmainframe product sizing Resource Value Units (RVU’s) – for z/OS only Virtual Processor Core License – for Db2 WH only “Per Install” target only licenses to add targets to existingreplication installations 36

Resources and Contacts37

Replication Professional Services and Training Architecture and Implementation Resilient, robust, scalable deploymentsHealthchecks Monitoring and automation Scripting Schema evolutionKim May kim.may@thefillmoregroup.com 410-465-6335Version Upgrades InfoSphere Data Replication v10.2.1EoS: 2018-04-30 InfoSphere Data Replication v11.3.0EoS: 2019-09-30IBM Authorized Training38 Change Data Capture Q Replication