Advanced Analytics And R/Python Integration

Transcription

Advanced Analytics andR/Python IntegrationUgo GiaquintoSenior Solution Architect

Legal DisclaimerQlik roadmaps provides a general overview of our anticipated product direction as of the date hereof. Qlikundertakes no intention or obligation to update or revise any forward-looking statements contained herein,whether as a result of new information, future events or otherwise.The information contained in Qlik roadmaps is the proprietary information of Qlik and is not to be used forany reason other than to provide our customers, prospective customers, partners and strategic partners witha general overview of our anticipated future offerings. Our roadmaps are not a commitment, promise orother obligation on the part of Qlik to deliver any particular code or functionality. Furthermore, Qlik makes nocommitment that any future functionality may be made available as part of maintenance and support.Roadmap information should not be relied on in making a purchasing decision, as the development,release, and timing of any features or functionality described for Qlik’s products remains at our solediscretion. 2017 QlikTech International AB. All rights reserved. Qlik , Qlik Sense , QlikView , QlikTech , Qlik Cloud , Qlik DataMarket , Qlik Analytics Platform , Qlik NPrinting ,Qlik Connectors , Qlik GeoAnalytics and the QlikTech logos are trademarks of QlikTech International AB which have been registered in multiple countries. Other marksand logos mentioned herein are trademarks or registered trademarks of their respective owners.

Introduction toAdvancedAnalytics

Advanced Analytics GoalsHappy CustomersRun the Business BetterEliminate Bad BehaviorsRexer Analytics, 2015 Data Science Survey; html4

Common Use CasesManufacturing & High TechPublic Sector Demand ForecastingQuality & Machine BreakdownWarranty AnalyticsCrime PreventionTransportation OptimizationRecidivismHealthcareRetail & Consumer Patient Admission ForecastingReadmission AnalysisClinical OutcomesMarket Basket AnalysisRetail Store SelectionProduct RecommendationCommunicationsLife Sciences Customer Segmentation/ChurnQuality/Process ControlNext Best Offer/ActionTerritory AlignmentCampaign ManagementDrug Discovery & EffectivenessFinancial ServicesEnergy & Utilities Credit Risk AnalysisLoan DefaultFraud DetectionCustomer ChurnProactive FailurePeak Demand5

What our customers are askingWe have investedhundreds of hours indeveloping algorithms inR, Python, or otherengines to support theirbusiness We are evaluating QlikSense and one of ourmain requirements is toinclude advancedanalytics with visualdiscovery We have an advancedanalytics use case, suchas fraud detection, salesforecasting, inventorymanagement, or priceoptimization Can I leverage this inQlik Sense?Is Qlik Sense right formy business?Can Qlik Sensesupport this?6

Key questions to consider How will you deliver advanced analytics to users who make decisions? How will these decision makers explore and interact with advanced analytics calculations? How can your organization get the most out of the data scientists you have invested in?7

A broad spectrum of analytics rvice AnalyticsBI / ReportingPrescriptiveAdvanced AnalyticsConsumersProducersBusiness UserCorporateReporting Prompts Static Output Table / ListOptimizationGuidedAnalytics Dashboard Selection Linkage Benchmarking Trends Growth Anomaly Bar / Line ChartBusiness AnalystAd-hocAnalysis Filtering Sorting Pivot Table Data ExportData istics/Algorithms Visualization Aggregation Association Comparison Search Data Stories Set Analysis Subtotals RollingAggregation Classification Segmentation Scripting DataGeneration Randomization Relative Time Distribution Regression Forecasting Correlation Clustering Histogram Box Plot SentimentModeling /MiningScenarioAnalysis Sampling Decision Trees Time Series Text Mining Neural Nets RandomForests EnsembleModels Factor Analysis Monte CarloData Preparation8

Qlik Sense offers native and third party istics/AlgorithmsModeling /MiningScenarioAnalysis Advanced Analytics Integration Native capabilities include statistical functions, interactive visualizations, and scripting formany advanced analytics use cases Advanced Analytics Integration offers engine-level sharing between Qlik Sense and thirdparty tools such as R and Python, for incorporating advanced calculation and machinelearning into analyses9

The old wayData ScientistBusiness AnalystBusiness UserData AcquisitionResults InterpretationAction StepsData PreparationModel CreationData SelectionModel SelectionResult GenerationResult Presentation10

Roles are changingData ScientistBusiness AnalystBusiness UserModel CreationData AcquisitionData SelectionGovernance & OversightData PreparationBasic ParameterizationAutomated Model SelectionResult GenerationAdvanced ParameterizationResult PresentationAction Steps11

The benefit – advanced insights for everyone Advanced analytics in Qlik Sense deliver powerful insights to business users– Data scientists build advanced models and calculations– Business decision makers can utilize them in the context of associative exploration– Analytics are calculated and visualized in real-time as the user explores, based onselected context12

AdvancedAnalyticsIntegration

Introducing Advanced Analytics Integration Direct integration with 3rd party advancedanalytics engines through server-side extensionAPIsAllows data to be directly exchanged between theQlik engine and external tools during analysis– Leverages Qlik’s Associative Engine to passrelevant data based on user contextFull integration with Qlik Sense expressions andlibrariesConnectors can be built for any external enginesOpen source Analytic Connections are availablein GitHub for R and Python(https://github.com/qlik-oss)Leverage the power of advancedanalytics calculations in Qlik SenseEtc.14

AAI Demonstration – UK Crime Statistics15

How it works1User interacts with app,making a selection or asearch6Combined hypercubeIs visualized for theuser in the app2Hypercube recalculatedby Qlik Engine to thenew context5Qlik engine combineshypercube with newdata3In-context data andscript sent to externalengine4External engine runsand sends results toQlik engine16

Example Expression SyntaxName of Extension to 3rd Party ApplicationFunction that says “Take this text and pass it on and let back end serve revaluate it”.1st Line of R script – loads a needed library.2nd Line of R script – extracts the seasonal trend component from time series dataData to pass from Qlik as ‘in s(q sumBirthsPerMonth, frequency 12, start c(1946,1))) seasonal', Sum([Births per month]) as sumBirthsPerMonth)

The participantsDevelopers Build and customizeextensions and connectors Contribute to open sourceprojectsAdministrators Deploy and secureextensions andconnectors Manage environmentsand infrastructureData Scientists Design and develop thirdparty algorithms for R,Python, etc. Support development of QlikSense apps with advancedanalyticsBusiness Analysts and Users Build Qlik Sense apps withadvanced analytics forbusiness use cases Perform business analysis18

What makes our approach uniqueQlik’s Associative Engine working with all third party engines Connectors can be built for any third partyengines, through open APIs Third party engines quickly processsmaller, user-specific data sets As the user explores, only a small set ofchosen and relevant data is sent Far more speed than conventional batchtechniques Results are instantly visualized for theuser, allowing for further exploration Results for each user are sent back to QlikSense in real-time19

AdvancedAnalytics UseCases

Financial ServicesFraud Detection Leverage pattern analysis comparing customer activitywith peer group behavior Utilize customer’s own past behavior to identify outlyingtransactions Bayesian learning, neural networks, fuzzy neural nets,and combinations of neural nets and rules, have beenextensively exploredAbnormalityExpert knowledge is integrated with statistical power andClassificationrule-learning programs to uncover indicators of fraudulentbehavior from a large database of customer transactionsClustering21

HealthcarePatient Admission ForecastingKnowing how many patients are likely to be admittedprovides a forward look on bed availability and flagspotential capacity issues in the days or weeks to come Qlik solution has taken customer from one of the worstperforming ER's to one of the best in the country All the forecasting calculations are performed insideQlik and get 85% accuracyTrendingRegressionForecasting is an important aid in many areas of hospitalmanagement, including elective surgery scheduling, bedClusteringmanagement, and staff resourcing22

Financial ServicesCredit Loss Forecasting Credit managers need to predict their expected futurecredit losses. This is mandatory for regulations like iFRS9. Qlik’s solution enables teams to ask complex questionswithout notice – and answer them in real time. Managers can predict the expected losses for specificproducts – loans written in any region, in any timeperiod, filtered by employment type or credit grade.to using a sales app atGatorade deMexicoAbnormalityEnable Credit Managers to forecast with accuracy –Classificationunderstanding what factors influence credit losses, andoptimizing the credit book to minimize future losses.Clustering23

Data Miningtechnique basedupon the theory thatif you buy a certaingroup of items, yourare more (or less)likely to buy anothergroup of itemsCorrelationRetailMarket Basket Analysis Driving actionable insight from customer purchasepatterns hidden within the mountain of data Leverage advanced analytics and algorithms toproduce statistics like support, confidence, and liftfor a deep dive into product affinities Quickly consume but also allow business users toask the inevitable “next question” that goesbeyond simple static reportingAssociationRetailers must understand consumer preferences and purchasepatterns to help provide more targeted offers and tailored assortments,Clusteringresulting in larger basket sizes and locking in higher margins24

Market Basket Analysis25

ConsumerCustomer Churn Classification (Decision Tree) algorithms to segmentcustomers as at-risk Analyze different ‘What-if’ scenarios by modifying theinput variables and assess the overall impact oncustomer models Evaluation and processing of call records for sentimentanalysis and categorizationSentimentFocus retention campaigns and deliver preferential servicesClassificationto at-risk high-value subscribers who have been predictedto have a propensity to churnAffinity26

Customer Sentiment Analysis27

For More Information Qlik Community Resources –Advanced Analytics Integration yticsintegration Qlik Product Innovation Blog tion28

ThankYou

analytics engines through server-side extension APIs Allows data to be directly exchanged between the Qlik engine and external tools during analysis – Leverages Qlik’s Associative Engine to pass relevant data based on user context Full integration with Qlik Sense expressions and