Transcription
Front coverAccelerating DigitalTransformation on ZUsing Data VirtualizationBlanca BordenCalvin FudgeJen NelsonJim PorellRedpaper
IBM RedbooksAccelerating Digital Transformation on Z Using DataVirtualizationDecember 2018REDP-5523-00
Note: Before using this information and the product it supports, read the information in “Notices” on page v.First Edition (December 2018)This edition applies to IBM Data Virtualization Manager for z/OS V1.1.This document was created or updated on December 18, 2018. Copyright International Business Machines Corporation 2018. All rights reserved.Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP ScheduleContract with IBM Corp.
ContentsNotices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vTrademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiAuthors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiNow you can become a published author, too! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiComments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiStay connected to IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiChapter 1. Business value of data virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.1 No time to wait for data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.1.1 Data virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.2 Unlocking mainframe data for an API economy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1223Chapter 2. Virtualization on IBM Z . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.2 Design, development, and testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.3 Any-to-Any connectivity for applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.3.1 Driver and API support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.3.2 Data source support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.4 DVM Runtime . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.5 Virtual Parallel Data enhancement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.6 Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.7 Management and control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.8 Use of specialty agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.9 Data management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Chapter 3. IBM Data Virtualization Manager in action . . . . . . . . . . . . . . . . . . . . . . . . . .3.1 API and z/OS Connect Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.2 Integrating cloud data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.2.1 Creating cloud applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.3 Optimizing ETL with Virtual Data Delivery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.4 Customer use case: Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.4.1 The problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.4.2 The solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.5 Customer use case: API and modernization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.5.1 The problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.5.2 The solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1314151516171717181819Chapter 4. Data Virtualization conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4.1 Look for tortured data flows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4.2 Consider how data is visualized . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4.3 Identify modern application development and operations capabilities . . . . . . . . . . . . .4.4 Operational benefits of data virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2122222223 Copyright IBM Corp. 2018. All rights reserved.iii
ivAccelerating Digital Transformation on Z Using Data Virtualization
NoticesThis information was developed for products and services offered in the US. This material might be availablefrom IBM in other languages. However, you may be required to own a copy of the product or product version inthat language in order to access it.IBM may not offer the products, services, or features discussed in this document in other countries. Consultyour local IBM representative for information on the products and services currently available in your area. Anyreference to an IBM product, program, or service is not intended to state or imply that only that IBM product,program, or service may be used. Any functionally equivalent product, program, or service that does notinfringe any IBM intellectual property right may be used instead. However, it is the user’s responsibility toevaluate and verify the operation of any non-IBM product, program, or service.IBM may have patents or pending patent applications covering subject matter described in this document. Thefurnishing of this document does not grant you any license to these patents. You can send license inquiries, inwriting, to:IBM Director of Licensing, IBM Corporation, North Castle Drive, MD-NC119, Armonk, NY 10504-1785, USINTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION “AS IS”WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITEDTO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR APARTICULAR PURPOSE. Some jurisdictions do not allow disclaimer of express or implied warranties incertain transactions, therefore, this statement may not apply to you.This information could include technical inaccuracies or typographical errors. Changes are periodically madeto the information herein; these changes will be incorporated in new editions of the publication. IBM may makeimprovements and/or changes in the product(s) and/or the program(s) described in this publication at any timewithout notice.Any references in this information to non-IBM websites are provided for convenience only and do not in anymanner serve as an endorsement of those websites. The materials at those websites are not part of thematerials for this IBM product and use of those websites is at your own risk.IBM may use or distribute any of the information you provide in any way it believes appropriate withoutincurring any obligation to you.The performance data and client examples cited are presented for illustrative purposes only. Actualperformance results may vary depending on specific configurations and operating conditions.Information concerning non-IBM products was obtained from the suppliers of those products, their publishedannouncements or other publicly available sources. IBM has not tested those products and cannot confirm theaccuracy of performance, compatibility or any other claims related to non-IBM products. Questions on thecapabilities of non-IBM products should be addressed to the suppliers of those products.Statements regarding IBM’s future direction or intent are subject to change or withdrawal without notice, andrepresent goals and objectives only.This information contains examples of data and reports used in daily business operations. To illustrate themas completely as possible, the examples include the names of individuals, companies, brands, and products.All of these names are fictitious and any similarity to actual people or business enterprises is entirelycoincidental.COPYRIGHT LICENSE:This information contains sample application programs in source language, which illustrate programmingtechniques on various operating platforms. You may copy, modify, and distribute these sample programs inany form without payment to IBM, for the purposes of developing, using, marketing or distributing applicationprograms conforming to the application programming interface for the operating platform for which the sampleprograms are written. These examples have not been thoroughly tested under all conditions. IBM, therefore,cannot guarantee or imply reliability, serviceability, or function of these programs. The sample programs areprovided “AS IS”, without warranty of any kind. IBM shall not be liable for any damages arising out of your useof the sample programs. Copyright IBM Corp. 2018. All rights reserved.v
TrademarksIBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business MachinesCorporation, registered in many jurisdictions worldwide. Other product and service names might betrademarks of IBM or other companies. A current list of IBM trademarks is available on the web at “Copyrightand trademark information” at http://www.ibm.com/legal/copytrade.shtmlThe following terms are trademarks or registered trademarks of International Business Machines Corporation,and might also be trademarks or registered trademarks in other countries.Bluemix CICS dashDB DB2 Db2 IBM IBM Cloud IBM MobileFirst IBM Z Informix InfoSphere QMF RACF Redbooks Redpaper Redbooks (logo)z/OS The following terms are trademarks of other companies:Linux is a trademark of Linus Torvalds in the United States, other countries, or both.Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States,other countries, or both.Java, and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or itsaffiliates.UNIX is a registered trademark of The Open Group in the United States and other countries.Other company, product, or service names may be trademarks or service marks of others.viAccelerating Digital Transformation on Z Using Data Virtualization
PrefaceThis IBM Redpaper publication introduces a new data virtualization capability thatenables IBM z/OS data to be combined with other enterprise data sources in real-time,which allows applications to access any live enterprise data anytime and use the power andefficiencies of the IBM Z platform.Modern businesses need actionable and timely insight from current data. They cannot affordthe time that is necessary to copy and transform data. They also cannot afford to secure andprotect each copy of personally identifiable information and corporate intellectual property.Data virtualization enables direct connections to be established between multiple datasources and the applications that process the data. Transformations can be applied, in line, toenable real-time access to data, which opens up many new ways to gain business insight withless IT infrastructure necessary to achieve those goals. Data virtualization can become thebackbone for advanced analytics and modern applications.The IBM Data Virtualization Manager for z/OS (DVM) can be used as a stand-alone productor as a utility that is used by other products. Its goal is to enable access to live mainframetransaction data and make it usable by any application. This this what? enablescustomers to use the strengths of mainframe processing with new agile applications.Additionally, its modern development environment and code-generating capabilities enableany developer to update, access, and combine mainframe data easily by using modern APIsand languages. If data is the foundation for building new insights, IBM DVM is a key tool forproviding easy, cost-efficient access to that foundation.AuthorsThis paper was produced by a team of specialists from around the world working around theworld.Blanca Borden is the WW IBM QMF and DVM Marketing Manager. Blanca has beeninvolved in mainframe software development, product management, marketing, and sales forover 25 years, including IBM DB2 , IMS, QMF, Data Virtualization, and the DB2 Tools. Shehas been in technical roles as well as management, marketing, and sales enablement.Calvin Fudge is Director of Product Marketing for Rocket Software’s data integration, API,and modernization portfolio. He has spent 20 years focused on helping bring to market newtechnologies in data integration, web services, and APIs. He is part of a team that helpedlaunch the industry’s first mainframe data virtualization solution. He writes frequently aboutthe modern mainframe and the foundational value of data for gaining a competitive advantagein the digital age. Calvin holds a Bachelor of Arts degree in Communication from theUniversity of Houston.Jen Nelson is a Senior Director of Software Engineering in Austin, Texas. For the last 20years she has focused on business intelligence and analytics, and is a recognized expert onhow high-powered analytics can transform organizations. She holds a BA in Political Sciencefrom the University of Texas at Austin. Copyright IBM Corp. 2018. All rights reserved.vii
Jim Porell is a Director of Solutions Advisors at Rocket Software in the United States. He has38 years of experience developing, designing and deploying mainframe solutions, as well asenterprise workloads that communicate with the mainframe. His area of expertise is hybridcomputing that includes mainframe systems. He has written extensively on security, opensource software and application development. He holds a Bachelor of Arts degree inMathematics from Providence.Now you can become a published author, too!Here’s an opportunity to spotlight your skills, grow your career, and become a publishedauthor—all at the same time! Join an IBM Redbooks residency project and help write abook in your area of expertise, while honing your experience using leading-edgetechnologies. Your efforts will help to increase product acceptance and customer satisfaction,as you expand your network of technical contacts and relationships. Residencies run fromtwo to six weeks in length, and you can participate either in person or as a remote residentworking from your home base.Find out more about the residency program, browse the residency index, and apply online at:ibm.com/redbooks/residencies.htmlComments welcomeYour comments are important to us!We want our papers to be as helpful as possible. Send us your comments about this paper orother IBM Redbooks publications in one of the following ways: Use the online Contact us review Redbooks form found at:ibm.com/redbooks Send your comments in an email to:redbooks@us.ibm.com Mail your comments to:IBM Corporation, IBM RedbooksDept. HYTD Mail Station P0992455 South RoadPoughkeepsie, NY 12601-5400Stay connected to IBM Redbooks Find us on Facebook:http://www.facebook.com/IBMRedbooks Follow us on Twitter:http://twitter.com/ibmredbooks Look for us on LinkedIn:http://www.linkedin.com/groups?home &gid 2130806viiiAccelerating Digital Transformation on Z Using Data Virtualization
Explore new Redbooks publications, residencies, and workshops with the IBM Redbooksweekly sf/subscribe?OpenForm Stay current on recent Redbooks publications with RSS x
xAccelerating Digital Transformation on Z Using Data Virtualization
1Chapter 1.Business value of datavirtualizationWe are living in a digital age where more data is being created than ever before. Industryresearch firm IDC estimates that by 2020, the digital universe will reach 40 zettabytes (ZB),which is 40 trillion GB of data, or 5,200 GB of data for every person on Earth.Paradoxically, this data is making it harder, not easier, for organizations to gain the insightsthey need to succeed. This data-insight dichotomy can be traced back to challenges ofaccessing heterogeneous data that is in many formats and platforms; that is, structured,unstructured, mainframe, distributed, on-premise, cloud, and mobile.Given the exponential increase in volume, it is increasingly difficult to physically assemble allof the data, let alone transform it into a homogenous structure that can be accessed by digitaland advanced analytics applications.In this chapter we describe the need to deliver data promptly and how to access data storedon the mainframe.This chapter includes the following topics: 1.1, “No time to wait for data” on page 2 1.2, “Unlocking mainframe data for an API economy” on page 3 Copyright IBM Corp. 2018. All rights reserved.1
1.1 No time to wait for dataWe no longer have the luxury of waiting for data. We live in a world where people expectinstant access to everything from restaurant reservations to sports scores, and they do notsettle for delays.Today, transactions must occur at the speed of a mouse click. Unfortunately, the traditionalmethod of managing data involves physically moving it to a central repository, such as anenterprise data warehouse that uses established tools that extract, transform, and load (ETL)data in a multi-step process that is expensive, complex, time-consuming, and CPU intensive(see Figure 1-1).Figure 1-1 Typical ETL processThese ETL architectures can dramatically affect the currency of information that is fed tocustomer and operational applications. This problem is critical for organizations that needinformation immediately because stale or inconsistent data is unacceptable for today’sapplications.1.1.1 Data virtualizationThe days of the physical data warehouse might well be numbered because too much datamust be moved in a timely, economical manner. Yet, comprehensive access to all enterprisedata is the goal of enterprises.The answer to a modern data warehouse might be in data virtualization. Data virtualization isa data integration method that brings together disparate data sources to create virtualized,integrated views of the data. These virtual views or tables are held in memory and available toapplications as logical data sources.Virtualization provides an alternative to physically moving data into a data warehouse orrepository to integrate and normalize the data structure. Instead, joining, combining, andnormalizing data is done in place in mainframe memory. No data movement is performed.Data virtualization also provides a data services layer that abstracts the underlying physicalimplementation of the individual data sources. Developers can manipulate and update datawithout needing specific skills or an understanding of the data structures.Data virtualization can be used to make mainframes into powerful homes for analytics of allenterprise data, which helps organizations improve transparency into their data to uncoverthe information th
This IBM Redpaper publication introduces a new data virtualization capability that enables IBM z/OS data to be combined with other enterprise data sources in real-time, which allows applications to access any live enterprise data anytime and use the power
