WIXLIB

The Simplest Way to Search Allof Your Big Data Sources,including IMS ;)Hélène LyonDistinguished Engineer & CTO, zAnalytics & IMS for EuropeIBM Systems, Software Sales,EuropeIMS Technical Symposium 2015 2015 IBM Corporation

Agenda Big Data in an “Data Driven” economy Why start with z Systems IMS strategies for big data Summary2 2015 IBM Corporation

Market Opportunities and ChallengesInvestments in InfrastructureBig DataReadycontinue to increase, with73% of organizationshaving invested orplanning to investwithin 24 months.(1) 10% organizations fullyprepared for newdigital trends likecloud, mobile, socialand analytics(CAMS)(2)CloudReady91% software will bebuilt for clouddelivery in 2014 (3)DataAnalyzed12% of the datacompanies alreadyhave, leaving 88% ofit on the cuttingroom floor (4)Sources1. Gartner Inc, Research Note G00263798 - Survey Analysis: Big Data Investment Grows but Deployments Remain Scarce in 2014 . NickHeudecker, Lisa Kart, Date Published 09/09/20142. IBM Institute for Business Value - Date published : 01/09/20143. IDC, “Directions 2014, Information is Everywhere”, Robert Mahowald, Date published: 17/03/20144. Forrester Research, Inc., “The Forrester Wave(TM): Big Data Hadoop Solutions, Q1 2014” Mike Gualtieri and Noel Yuhanna,Datepublished: 27/02/20143 2015 IBM Corporation

Why is it happening?CloudprivateMobilepublicSaaSData is Leaving the Data Center Stored on shared drives Hosted by 3rd party Managed by 3rd partyConsumerization of ITBigDataBYODAppsSocialData is Generated 24x7 Used Everywhere Always Accessible On private devicesEverything isEverywhereHadoopNo-SQLFilesData is Produced in high volumes Stored unstructured Analyzed faster/cheaper MonetizedData Explosion There is more data Data is leaving the data center Data is consumed everywhere Data is worth more than ever before4 2015 IBM Corporation

New paradigms impacting Big Data & AnalyticsZBs oftransactiondataCloudSocialMobileInternet ofThings5 2015 IBM Corporation

We’ve moved into a new era of computing Radical Flexibility12 terabytesExtreme Scalabilitymillion5of Tweetscreate daily100’strade eventsper secondVolumeVelocityVarietyVeracityOf video feeds fromsurveillance camerasInformation from everywhereOnly1 in 3Decisionmakers trusttheir information“We have for the first timean economy based on akey resource[Information] that is notonly renewable, but selfgenerating.Running out of it is not aproblem, but drowning init is.”– John Naisbitt6 2015 IBM Corporation

And Analytics is becoming the Keystone of every organization Analytics derive insight from data– To help optimize business performance– To build new innovative services– To fight against fraud– To make all customer interaction personal!– Analytics become Business Critical!– Analytics services are tightly integrated with business critical applications and data Often hosted in z/OS transaction and batch systems Often relying on copies or aggregation of transaction and application data– Analytics is part of the flow of the business.– Decision processes have to be improved with new business insight derived from realtime or near real time data.– Failure of these applications for any length of time can result in lost business orreputation.– Analytics solutions need to support a large concurrent user population with high volumesof requests. Analytics are only as good as the underlying data foundation– Data governance & Security & Performance7 2015 IBM Corporation

Agenda Big Data in an “Data Driven” economy Why start with z Systems IMS strategies for big data Summary8 2015 IBM Corporation

The Big Data Starting PointWhich typesof big datadoes yourorganizationcurrentlyanalyze?Source : Gartner Inc,Research Note G00263798Survey Analysis: Big DataInvestment Grows butDeployments Remain Scarcein 2014 . Nick Heudecker,Lisa Kart, Date Published09/0920149 2015 IBM Corporation

The z Systems advantage10 2015 IBM Corporation

Imagine the possibility of leveraging all of your data assetsTraditionalApproachStructured, analytical,logicalData: Rich, historical,private, structuredCustomers, history,TransactionsThe “Circle ofTrust””New ultimediaWeb LogsInternal AppDataMainframeDataStructuredRepeatableLinearOLTP SystemDataData warehouse & businessanalytics moving closer tothis dataNew ideas,new questions,new answersSocial DataUnstructuredExploratoryDynamicText Data:emailsSensor data:imagesERPDataCreative, holisticthought, intuitionData: Intimate,unstructured.Social, mobile,GPS, web, photos,video, email, logsRFIDTraditionalSourcesNewSourcesThe real benefit is derived from integration of new data sources with traditional corporate data How can you query across both realms? How can you preserve security and lower TCO? How can you avoid costs and risks of offloading?11 2015 IBM Corporation

to deliver Improved Business Outcomes1. Enrich your information basewith Big Data Exploration2. Improve customer interactionwith Enhanced 360º View of the Customer3. Optimize operationswith Operations Analysis4. Gain IT efficiency and scalewith Data Warehouse Augmentation5. Prevent crimewith Security and Intelligence Extension12 2015 IBM Corporation

Unfortunately for most of our clients, their data lifecycle is toofragmented to gain advantage from that data Client key concerns:– Cannot deliver real-time analytics– Inadequate performance– Governance model– Data latency– Data completeness Not all in one source Lack access to fine-grained data Lack “customer intent” e.g. cancelledtransactions– Multiple platforms, many securityboundaries, many points of failure,– Challenging recovery scenarios Multi-day workshop captured thecomplexity of the current architecture The picture does not show all the stepsbefore the data gets to the off-platformwarehouse13 2015 IBM Corporation

Agenda Big Data in an “Data Driven” economy Why start with z Systems IMS strategies for big data Summary14 2015 IBM Corporation

Analytics on IMS Databases & SystemsUse case BI, dashboarding,reporting of IMS data Merge HDFS data withtrusted OLTP IT analytics (IMS log data)15Solution QMF Cognos 10.2 BI IBM InfoSphereBigInsights Bring analytics to thedata IBM DB2AnalyticsAccelerator Visualize entire bigdata landscape IBM WatsonExplorer 2015 IBM Corporation

Analytics in IMS ApplicationsUse case Predictive analytics Decision management Fraud detectionSolution SPSS OperationalDecision ManagerSeeSessionC14 HLyon16 2015 IBM Corporation

QMF for z/OS offers fast, simple connection to a broadspectrum of data sourcesSeeSessionA14 SMinkIBM DB2 IBM DB2InformixTeradatafor iSeries for ess–httpdataIBMPureDataIMSIBM DB2for LUWQMF 11CloudantBigInsightsSun JDBCODBC bridgedata sourcessuch asExcel andtext files etc.SolidDBHadoop 2015 IBM Corporation

QMF for z/OS 11 and IMS solution highlights Maximize benefits from IMS environment– Allow users to graphically construct ad-hoc IMS queries– Create reports and dashboards that draw directly from IMS data– Roll out web-based graphical content that blends IMS data with relational, andunstructured data sources– Accessible from all current interfaces,web, workstation, tablets– IMS support is available with QMF for z/OS. QMF Classic Edition does not support IMSas a data source zIIP usage model– QMF JDBC workloads are zIIP-eligible18 2015 IBM Corporation

Cognos 10.2 BI with IMS Data Certified against IMS 12 using IMS Open Database technology– Universal JDBC driver Real-time analyticsGet Started Today! developerWorks article available here19 2015 IBM Corporation

Enhancing IMS analytics on Linux on z with Big Data Much of the world’s operational data resides on z/OS Unstructured data sources are growing fastSeeSessionC08 KDurward There is a need to merge this data with trusted OLTP data from z/OS data sources IMS provides the connectors and the DB capability to allow BigInsights to easily andefficiently access the IMS data sourceAnalysisJDBCDiscoveryHDFS20 2015 IBM Corporation

IBM InfoSphere BigInsights on System zIBM InfoSphere System z Connector for Hadoop Leverage the power of Hadoop on themainframe Drag-and-drop extracts from mainframesources Protect sensitive data Faster application delivery Seamless interoperabilityAll "z" deployment for maximum security21 2015 IBM Corporation

IBM InfoSphere Data Replication FamilyExpand z/OS data integration with changed data ONLY feedsz Systemz/OSLinux for System zBigInsightsDB2VSAMData logs Non-intrusive Changed DataCapture and Delivery– Log-based capture minimizes impacton source application environment– Dramatically reduces volume of data only the changes not the entire source– Continuous or Periodic delivery withconfigurable switch of HDFS target– One model for z/OS and distributeddata (DB2 on all platforms, IMS,Oracle, )– No need to take sources "off line"– Fully recoverable– Native HDFS ApplyInfoSphere DataReplicationMapReduceDeliver only the ChangesHbase,HiveSMIMSFLogsz/VMCP(s)IFLIFL IFL NOT for system/application logs andother sequential sources – These are always full file transfers!22 2015 IBM Corporation

IT Analytics for IMSCustom ApplicationsHealth CareNetworkingInsuranceShrink Wrap Solutions“x2020”Telco“Unity” Log ingestion and analysis Value: allow correlation of an entireecosystem of application servers withIMS to provide deep insight, filtering,analytics, as well as faceted searchcapabilitiesIBM Big Data PlatformMDA AcceleratorTools Client Specific Customizations, Visualization tools (“zInsights”)DomainSpecificTelcoFinancial servicesRetailHealthcareGenericParsers and ExtractorsFederated Discovery, PatternDiscovery,Search, Visualization Tools(applications, services,forrootcause analysisservers and devices )IBM Big Data sualization& celeratorsInformation Integration & Information Integration & Governance23 2015 IBM Corporation

IMS and IBM Accelerator for Machine Data Analytics Consume log data produced by Transaction Analysis Workbench Index and link transactions together across products (IMS, DB2, MQ, CICS, WAS) Make large amounts of IMS transactional log data available to the suite of BigInsightstools.BigInsights PlatformHDFSMDATransaction Analysis WorkbenchIndex, ExtractionSFTPSearch in Watson ExplorerBigSheetsLog Conversion to ASCII in CSV format24 2015 IBM Corporation

Watson Explorer - visualization and discovery across all datasources: “Integration at the glass”WatsonExplorerSecurely connect toand leverage datastored in DB2 forz/OS & IMSProviding unified, real-timeaccess and fusion of bigdata unlocks greaterinsight and ROIImprove customerservice & reducecall timesIncrease productivity &Analyze customer informationleverage past work& data to unlock trueincreasing speed to marketcustomer value25Help prioritize your zSystem big dataintegration andanalytics projectsCreate unified view ofALL information forreal-time monitoringIdentify areas of informationrisk & ensure datacompliance 2015 IBM Corporation

IMS Watson Explorer Configuring the IMS source– After deploying the IMS JDBC driver, create a new Database seed Setting up the data transformation– After creating a new seed, a converter needs to be configured using standard XPATH26 2015 IBM Corporation

Original IMS hierarchy for hospital database Hierarchy: HOSPITAL- WARD- PATNAME Goal: Get a patient centric view27 2015 IBM Corporation

Why use Watson Explorer Previously to change the schema so that the PATIENT information is at the top, alogical database needs to be created This requires a DBA to be involved and a time window when the new databaseresources can be brought online Watson Explorer allows indexes to be created dynamically and for better searchingthat is not restricted to IMS Segment Search Arguments (SSAs)28 2015 IBM Corporation