Value Of Moving Streaming Analytics Outside Data Center

1y ago

30 Views

1 Downloads

6.51 MB

39 Pages

Report/dmca

Download PDF

Transcription

The Value of Moving StreamingAnalytics Outside the Data CenterMark LochbihlerDirector, Partner EngineeringHortonworks#AnalyticsXC o p y r ig ht 201 6, S A S I n st i t ut e I n c. A l l r ig ht s r ese rve d.

DisclaimerThis document may contain product features and technology directions that are underdevelopment, may be under development in the future or may ultimately not be developed.Project capabilities are based on information that is publicly available within the Apache SoftwareFoundation project websites ("Apache"). Progress of the project capabilities can be tracked frominception to release through Apache, however, technical feasibility, market demand, user feedbackand the overarching Apache Software Foundation community development process can all effecttiming and final delivery.This document’s description of these features and technology directions does not represent acontractual commitment, promise or obligation from Hortonworks to deliver these features in anygenerally available product.Product features and technology directions are subject to change, and must not be included incontracts, purchase orders, or sales agreements of any kind.Since this document contains an outline of general product development plans, customers shouldnot rely upon it when making purchasing decisions.1 Hortonworks Inc. 2011 – 2016. All Rights Reserved

Your PresenterMark LochbihlerHortonworks Partner Engineering@MarkLochbihler“26 years of Experiencein Computer Science, SAS and Data Platforms”Page2 2 Hortonworks HortonworksInc. 2011 – 2016.Inc. 2011All Rights– 2015.ReservedAll Rights Reserved

Today’s AgendaSeptember 14th, 2016Hortonworks and SAS Partnership Data Explosion, the Market and Joint Customer StoriesHortonworks Connected Data Platforms Hortonworks Data Platform Hortonworks Data Flow Hortonworks and SAS Integrations (High Level Overview) Focus on SAS ESP Integrations with HDP and HDF SAS ESP running in HDP SAS ESP running with HDFEdge Analytics Value Summary 3 Hortonworks Inc. 2011 – 2016. All Rights Reserved

Founded in 2011Original 24 architects, developers,operators of Hadoop from Yahoo! 800 customers (as of Jan 1 st, 2016) Publicly traded on NASDAQ: HDP The Leader in Connected Data PlatformsData in Motion - Hortonworks Data FlowData At Rest - Hortonworks Data PlatformPowering Modern Data Applications Leader in open-source community, focused on innovation to meet enterprise needs Unrivaled Hadoop support subscriptions5 Hortonworks Inc. 2011 – 2016. All Rights Reserved800 1500 EMPLOYEESECOSYSTEMPARTNERS

SAS Hortonworks Global Alliance Strategic alliance established October 2013 Dedicated Alliance Management Tier-1 Hadoop Distribution Vendor for SAS"The expanded integration of SAS w ithHortonw orks Data Platform provides a simplew ay for customers to broaden their analyticoperations across new data sets that candrive smarter business decisions."Shaun Connolly , VP of Corporate Strategy ,Hortonworks Joint R&D with YARN integration Joint Product Roadmap Both – Founding Members of ODPi and DGI”Adopting YARN allow s us to use the YARNinfrastructure to set the boundaries for theprocesses needed to run SAS HPA productsand SAS LASR Analytic Server basedproducts. CPU and memory can be capped,facilitating a better sharing model for thecluster."Paul Kent, Vice President, Big Data, SAS6 Hortonworks Inc. 2011 – 2016. All Rights Reserved

44ZB 4ZBINTERNETOFANYTHINGDATATOMORROW8 Hortonworks Inc. 2011 – 2016. All Rights ReservedDATA

Data Powers Highway SafetyPrecipitationClick-streamLocationServer logSensor9 Hortonworks Inc. 2011 – 2016. All Rights ReservedSocialMobileTire Pressure

Data Powers Better HealthEMR DataWearable DevicesClick-streamServer logsSensor10 Hortonworks Inc. 2011 – 2016. All Rights ReservedMedical ResearchClaims CodesMobile

Data Powers Digital SecurityVirus DefinitionsFirewall LogSocialSensorServer Log11Click-stream Hortonworks Inc. 2011 – 2016. All Rights ReservedMobileEmails

Customer Insights – SAS leveraging a centralized Big Data LakeTelco / Media Large multi-channelmedia providerChallenge Unable to analyze huge amounts of data to optimize and improve real -time customer insights Understand audience: Having the largest volume of data sets, audience segments/profile while leading the marketplace inprivacy and governance. Find Audience: Being leaders in identifying and targeting audiences across channels, platforms and devices. Engage Audience: Driving engagement across platforms and formats. Measure Audience: Exceeding client expectations with transparent reporting and accurate attribution models.SolutionWhy Hortonworksand SAS?Customer Insight12 Integration and analysis of all data collected across the organization Query ALL data in one location blend of online and offline data, subscription, ecommerce, loyalty programs, etc. Land massive click stream log files, 100 M records / day, 30 million unique IDs / month Use 100% of the data for analysis and visualization instead of smaller random samples (over sampling) Identified and modeled more than 600 relevant web characteristics out of a field of 75,000 with SAS Hortonworks Inc. 2011 – 2016. All Rights Reserved

Unified Customer Record - 360 Customer View - to Improve SalesChallengeRetailMajor homeimprovement retailer Lack of unified customer record across all channels clouded targeting for marketing campaigns No “golden record” for analytics on customer buying behavior across all channels Data repositories on web traffic, POS transactions and in-home services were in silos Data storage costs were increasing, without a corresponding increase in valueSolutionWhy Hortonworksand SAS?Single View13 HDP data lake drives golden customer record, targeted marketing, and reduction in data storage expenses Golden record enables targeted, personalized marketing with higher success rates Data warehouse offload saved millions of dollars in recurring expense Price optimization versus competitors several millions in top-line revenue growth Hortonworks Inc. 2011 – 2016. All Rights Reserved

Improve Reimbursement - by Finding Errors in ClaimsInsuranceHealthcareLarge US medicalinsurerChallenge Difficulty identifying coding errors among 300K daily claims Health insurer had goals of marrying electronic health records with claims data Data analysis is disjointed it difficult to identify coding errors Undiscovered errors may harm patient health and reduce reimbursement from government programs, costing manymillions in missed paymentsSolutionWhy Hortonworksand SAS? Using SAS and Hortonworks to improve reimbursement revenue and health outcomes HDP SAS: Marrying and analyzing numerous pool of data store in HDP —including gross margins, taxes, customer claimsand policy premiums—to determine the company's potential exposure and manage its resources more effectively. Ability to crunch several terabytes of data, and then revises, recalculates and reports on that data on a weekly basis.Data Discovery14 Hortonworks Inc. 2011 – 2016. All Rights Reserved

Hortonworks DeliversConnected Data PlatformsModern Data NG16 Hortonworks Inc. 2011 – 2016. All Rights ReservedPERISHABLEINSIGHTSHISTORICALINSIGHTSDATA INMOTIONDATA AT RESTHortonworksDataFlowHortonworksData Platform

The Value of Modern Data AppsCustom or Off the ShelfModern Data ApplicationsReal-Time Cyber Securityprotects systems with superior threatdetectionSmart Manufacturingdramatically improves yields by managingmore variables in greater detailACTIONABLEINTELLIGENCEConnected, Autonomous Carsdrive themselves and improve road safetyDATA INMOTIONDATA ATRESTFuture Farmingoptimizing soil, seeds and equipment tomeasured conditions on each square footAutomatic Recommendation Enginesmatch products to preferences in millisecondsHortonworksDataFlow17 Hortonworks Inc. 2011 – 2016. All Rights ReservedHortonworksData Platform

Central Management of Data at RestYAR NDATA OPERATING SYSTEMCentralized Platformfor operations, governance and C EOPERATIONSStreamingSearch19Maximum Data Ingestincluding existing and new sources,regardless of raw formatSECURITYInteractive Hortonworks Inc. 2011 – 2016. All Rights ReservedDiverse Applicationsrun simultaneously on a single clusterShared Big Data Assetsacross business groups, functionsand users

HDF Provides “Data Plan of Control” by Managing IoT DataflowsData source agnostic collection of data across heterogeneous environmentsConstrainedHigh-LatencyLocalized Context21 Hortonworks Inc. 2011 – 2016. All Rights ReservedHybrid – Cloud/On-PremiseLow -LatencyGlobal Context

Hortonworks AND SAS Deliver Advanced Analytics Anywhere!Data ManagementESPESPData DiscoverAdvanced AnalyticsGrid ManagerLASRHP ProceduresCode AcceleratorData Quality AcceleratorEPData Mining25 Hortonworks Inc. 2011 – 2016. All Rights Reserved

The Value of Event Stream ProcessingSAS ESP compliments HDF and HDP by offering“Complex Event Processing” or CEPCyber Security - identify a malicious intrusion before or as it occursFraud – analyze streaming transactions to determine which needs immediate attentionPredictive Maintenance – predict outlier conditions from streaming machine and sensor dataCustomer Experience and Marketing – use streaming data insights to personalize interactionsStream Data Management – transform and clean data in motion, storing only what you need27 Hortonworks Inc. 2011 – 2016. All Rights Reserved

Data Center Approach: SAS ESP processing is co-located within HDPGROUP 2GROUP 1HDPINTERNETOFANYTHINGGROUP 3ESPSTORAG ESTORAG EDATA AT RES TGROUP 4In this deployment model - SAS ESP provides “Complex Event Processing”at the point of data being ingested into Hadoop28 Hortonworks Inc. 2011 – 2016. All Rights Reserved

ResourceManager1) ESP Job LauncherSAS ESP on HDP is YARN ReadyESP Job Launcher2Scheduler2) Request ESP: Memory / Core iner 1.1NodeManagerAM23a) ESP r 1.2AM 1NodeManagerContainer 33b) ESP ServerNodeManagerNodeManagerContainer 1.329 Hortonworks Inc. 2011 – 2016. All Rights ReservedNodeManagerNodeManagerAM4Container 43c) ESP Server

Extending Streaming Analytics with SAS ESP and HDFExecute SAS ESP Advanced Analytics as a part of any HDF workflow : As data moves between data centers As data moves from the Edge or Remote Access Points to a data center As data moves from a data centers to the cloudRemote to Data CenterBetween Data CentersESPESPBetween Data Centers & CloudESPESPHDFHDFHDFESPHDFHDFHDFHDFESP30 Hortonworks Inc. 2011 – 2016. All Rights ReservedESP

SAS ESP can provide “Complex Event Processing” to any HDF workflowESPGROUP 2GROUP 1OFDA TA IN M OTIONGROUP 3ESPSTORAG EINTERNETANYTHINGSTORAG EDATA AT RES TGROUP 4SAS Models that were built on Historical Data Can be moved closer tothe “Edge” of a modern data application.31 Hortonworks Inc. 2011 – 2016. All Rights Reserved

Edge Analytics - On Premise and In the CloudCLOUDGROUP 1GROUP 2HDPSTORAGESTORAGEDATA AT RESTESPGROUP 3INTERNETOFMachineLearningGROUP 4ESPANYTHINGON PREMIS EGROUP 1GROUP 2HDPAnd, it should be noted that with SAS and HDP organizationsare executing Machine Leaning and Deep Historical ClosedLoop Analytics in the Cloud and Data Center as well.32 Hortonworks Inc. 2011 – 2016. All Rights ReservedGROUP 3ESPSTORAGESTORAGEDATA AT RESTDeepHistoricalAnalysisGROUP 4

SAS ESP Nifi Processors – “Drag and Drop” IntegrationSAS ESP Nifi Processorsenable seamless “Drag andDrop” integrationwithin any HDF WorkflowSAS ESP Nifi Processors comeswithin SAS ESP 4.1 which went GAin September 201633 Hortonworks Inc. 2011 – 2016. All Rights Reserved

HDF and SAS ESP can extend outside the Data CenterREGIONAL AND CORE INFRASTRUCTURESSOURCESSources or“Things”Sensors &ActuatorsPeople, Planes,Cars, Machines,Buildings, .Streaming Analytics andComputingEdge Gateways wData Aggregationand Filtering Data Flow ManagementSimple Event ProcessingComplex Event ProcessingRegional and Central DataCenters & the Cloud Event Stream ProcessingScalable StorageBig Data AnalyticsClient LibrariesMiNiFiESPESPESPHDF and SAS ESP Extend Beyond Traditional Data Center Firewalls:HDF offers Data Plan of Control – HDF offers Apache Nifi, Minifi and Client Libraries, Minifi and Nifi 34SAS ESP compliments HDF by providing “Complex Event Processing” anyway along the “Data Plan of Control” Hortonworks Inc. 2011 – 2016. All Rights ReservedESP

SummaryWhy Move Streaming Analytics Further out to the Edge?Reacting immediately to an important “Event” closer to the “Edge” can yieldsignificant positive results by: Increasing Customer Loyalty and Revenue Example: Providing a personalized, appealing offer that generates a close.Reducing Operational Inefficiencies and Expenses Examples: Stopping Fraud as it occurs instead Catching an early warning which alerts for immediate maintenance Only storing relevant events Enriching data as it is ingestedHDF and SAS ESP are integrated to allow an organization to implement specificedge streaming usage cases to improve bottom line results.36 Hortonworks Inc. 2011 – 2016. All Rights Reserved

#AnalyticsXC o p y r ig ht 201 6, S A S I n st i t ut e I n c. A l l r ig ht s r ese rve d.

ESP ESP ESP Remote to Data Center HDF HDF HDF Extending Streaming Analytics with SAS ESP and HDF ESP ESP ESP ESP Execute SAS ESP Advanced Analytics as a part of any HDF workflow : As data moves between data centers As data moves from the Edge or Remote Access Points to a data center As data moves from a data centers to the cloud HDF .