Transcription
DMM300 - SAP Business WarehouseData Provisioning from SAP and Non-SAP SourcesRainer UhleProduct Management SAP EDW (BW/HANA)Public
AbstractThe operational data provisioning (ODP) framework has been greatly enlarged during the last releases ofSAP Business Warehouse (BW) and offers new, optimized provisioning capabilities. Together with the SAPLandscape Transformation replication server implementation into ODP, fascinating real-time insights arenow supported wherever needed.
DisclaimerThis presentation outlines our general product direction and should not be relied on in making apurchase decision. This presentation is not subject to your license agreement or any other agreementwith SAP. SAP has no obligation to pursue any course of business outlined in this presentation or todevelop or release any functionality mentioned in this presentation. This presentation and SAP'sstrategy and possible future developments are subject to change and may be changed by SAP at anytime for any reason without notice. This document is provided without a warranty of any kind, eitherexpress or implied, including but not limited to, the implied warranties of merchantability, fitness for aparticular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in thisdocument, except if such damages were caused by SAP intentionally or grossly negligent. 2014 SAP SE or an SAP affiliate company. All rights reserved.Public3
SAP Data Management PortfolioEnd-to End Data Management & App Platform for Real-Time BusinessREAL-TIME APPLICATIONSConsumerEngagementSense &RespondSAP ASEREAL-TIME ANALYTICSPlanning &OptimizationOperationalAnalyticsSAP HANA PLATFORMeReal-time transactions end-to-end analyticsExtended Application ServicesBig DataWarehousingPredictive, Spatial &Text AnalyticsSAP SQLAnywhereProcessing EngineSAP ESPSAP HANA platformSAP IQDatabase ServicesReplicationServer 2014 SAP SE or an SAP affiliate company. All rights reserved.Application Function Lib. & Data ModelsIntegration ServicesSAP DataServicesPublic4
AgendaIntroduction – Data Provisioning Aspects regarding SAP Business Warehouse Traditional understanding Provide BW Business Data for Big Data ScenariosBW Operational Data Provisioning (ODP) Framework Operational Delta Queue (ODQ) for classic SAP ERP Extractors Realtime Replication via SLT and ODQProviding BW Business Data for Big Data Scenarios SAP HANA Spatial Analytics SAP HANA Sentiment Intelligence RDS 2014 SAP SE or an SAP affiliate company. All rights reserved.Public5
DMM300 - SAP Business WarehouseData Provisioning from SAP and Non-SAP SourcesIntroduction – Data Provisioning Aspects regarding SAP Business Warehouse Traditional understanding Data movement into one central repository (ETL processing, Replication) for good reasons Involve BW Business Data into Big Data Scenarios BW Business Data and HANA Spatial Services Text Analysis results for BW purposes
BW 7.4 – Architecture OverviewAvailable with BW 7.4, SP5/SP6 on HANAEnhanced Data Modeling New overall reference Architecture LSA BW/HANA Smart Data Access providing the logicalEDWBW Queries, ODATA, MDX, SQL Easy integration of external data models withOpen ODS LayerBW Virtual Datamart Layer Composite Provider for enhanced support of“Mixed Scenarios”Architected DatamartLayer Common Eclipse based Modeling environmentTransformationsPush down further processing logic to HANAEDW LayerTransformationsOpen Operational DataStore LayerHANA tables, views(Field based) DSOBW SchemaHANA Schema BW Analytic ManagerAgileDatamartsBWWorkspaces HANA Analysis Processes BW TransformationsNew class of Data Provisioning BW enhancements for Operational DataProvisioning (ODP) and Operational DeltaQueue(ODQ) PSA layer obsolete during Data Acquisition Simplified SLT handling in BWExternal Sources 2014 SAP SE or an SAP affiliate company. All rights reserved.Public7
Gartner‘s Logical Data Warehouse Reference FrameworkData ProvisioningBW Core PersistencyOpen ODS Layer Open ODS Views Staging ETL/ReplicationMixed Scenario Support HANA Model generation forBW InfoProvider 2014 SAP SE or an SAP affiliate company. All rights reserved.Public8
Operational Data Provisioning (ODP) InfrastructureSLTSAP DataServicesProviderSAP ERP ExtractorsOperationalData ProvisioningODQSubscriberTarget SAP BWHANA ViewsEmbedded AnalyticsSource SAP BW 2014 SAP SE or an SAP affiliate company. All rights reserved.Public9
SAP HANA SpatialThe challenge: Silos of information create an incomplete picture BusinessApplicationsGeographic InformationSystems(GIS)EngineeringSystems Transactional data Geographical data Diagrams Master data Location-based data 2D/3D graphs Analytical data Maps and topologies Animations 2014 SAP SE or an SAP affiliate company. All rights reserved.Public10
SAP HANA Sentiment Intelligence RDSCurrent Customer Situation Lack of insight into online channels like socialmedia, wikis, blogs, forums,, trader sites and more Can’t or extra effort to combine unstructured socialmedia data with CRM data or other structuredinternal dataValue Proposition Understand the local demand for products andservices Evaluate impact of marketing campaigns andevents Get early warning of product defects and shortfalls Channel and market-specific customer concernsand delights 2014 SAP SE or an SAP affiliate company. All rights reserved.Public11
DMM300 - SAP Business WarehouseData Provisioning from SAP and Non-SAP SourcesBW Operational Data Provisioning (ODP) Framework Operational Delta Queue (ODQ) for classic SAP ERP Extractors and SLT Replication ODP infrastructure (with delta queues) takes over important services Data is stored in a compressed state Data is retained in the delta queue for a specified time periodfor recovery purposes Realtime Replication from external source systems via SLT and ODQ
ODP/SLT Scenario for SAP Business WarehouseValue PropositionIf you use operational data provisioning, you can load the datadirectly into the InfoProvidersbypassing the PSA layer by using Data Transfer Process (DTP).*The ODP infrastructure (with delta queues) takes over importantservices such as monitoring data requests.The data is stored in a compressed state in the delta queue.A delta request transfers data records from the queue to thesubscriber.The data changes to a queue can also be requested by more thanone subscriber.The data is retained in the delta queue for a specified time periodfor recovery purposes. 2014 SAP SE or an SAP affiliate company. All rights reserved.* as of SAP BW 7.40Public13
HANA BW – Operational Data Provisioning (ODP) InfrastructureOverview about all new ODP based scenarios for BW 7.40Subscriber /ConsumerProviderMain use cases available withBW 7.40:1.ODP based Data Provisioning Aspects forSAP ERP SourcesODQ2.SLT/ODPSupportedSLT-Sourcebased real-time replication3.ODPbased data transfer between BWsystemsSLTOperationalDataProvisioning ODPis a NetWeaver based Framework, ODQas persitency resides where the ODP isinstalled as sourceODQTarget BW Eg. If SLT is installed within the target BW theODQ would of course also reside thereSAP ERP ExtractorsODQSource BW 2014 SAP SE or an SAP affiliate company. All rights reserved.Public14
New explicit ODP Source System Types in BW 7.40DTP w/o PSADTP w/o PSADSOODP DataSourceInfoCubeOperationalDelta QueueMasterDataDSOSAP BWSAP BWODP DataSourceOperational Delta Queue (ODQ)Extractor(BW SAPI DataSources)TableERM 2014 SAP SE or an SAP affiliate company. All rights reserved.SLTTableSAP ERP SourcesPublic15
Business Content DataSources/Extractors: Customer Balances a never ending success story!The extractor itself supports different options:Delta handling in various waysDirect Access capabilitiesReal-time Data Acquisition (RDA)ODP/ODQ Readiness 2014 SAP SE or an SAP affiliate company. All rights reserved.Public16
SLT based ABAP to ABAP Replication – Steps to be executedTo start the replication for a table you have to enter transaction LTRC and execute the Data Provisioning function. 2014 SAP SE or an SAP affiliate company. All rights reserved.Public17
Operational Delta Queue (ODQ) Monitor - SAP ERP ExampleODP DataSource becomes visible in Operational DeltaQueue with the request processing (InfoPackage/DTP) inthe subscriber systemReal-time mode can be switched on per subcriber 2014 SAP SE or an SAP affiliate company. All rights reserved.Public18
SAP ERP ODQ Monitor (Transaction ODQMON)Example from SAP ERP: ODQ in action 2014 SAP SE or an SAP affiliate company. All rights reserved.Public19
Given SAP ERP Business Content DataSourceODP capable ERP extractors can be replicated undersource system type ODP – SAP (Extractors) 2014 SAP SE or an SAP affiliate company. All rights reserved.Public20
BW Dataflow with DTP Extraction from ODP/ODQ 2014 SAP SE or an SAP affiliate company. All rights reserved.Public21
Simplification – Advanced DataStore ObjectNew with SAP BW 7.4 SP8 – OverviewActive Data(Reporting)Change Log(Delta Store)Next generation of Data Store Object simplifiespersistence management in SAP BW Combine InfoObject and field based modelling One type of InfoProvider with different settings toconsolidate DataStore Objects and InfoCubes New intuitive eclipse based modelling UIActivation Queue(Inbound Layer)Advanced DSOSAP BWSAP HANA 2014 SAP SE or an SAP affiliate company. All rights reserved.Optimized DataStore services High frequent and mass volume data loads –based on optimized request management Change of usage scenario without deletion of data Up to 120 key fields Optional SID entries for query performance optimizationPublic22
ODP for data extraction from the SAP Business Suite –Results from internal performance testingODP allows to skip the PSA layer and load directly with DTP from the source system into a DSO Runtime is reduced by more than 40%o Scenario: loading from the Operational Delta Queue (TA ODQMON) in the source system via DTP into a DSO comparedto loading from BW Service API Delta Queue (TA RSA7) via InfoPackage into a PSA and then via DTP into a DSO Throughput of 35 Mio records per hour is achieved w/o tuning (three times parallel processing)ODP doesn’t change the implementation of application extractors If the extractor is the ‘bottle neck’ the throughput won’t change 2014 SAP SE or an SAP affiliate company. All rights reserved.Public23
ODP enabled ExtractorsRequirementsProviderProviderThe ODP interface you must use one of the following releases of ERP and PI BASIS (or higher) in your ODQsystem (e.g. ERP system as source system): PI BASIS 2005 1 700 SP 24 (part of SAP NetWeaver 7.00 SP 24)PI BASIS 2006 1 700 SP 14PI BASIS 701 SP 9 (part of SAP NetWeaver 7.01 SP 9)PI BASIS 702 SP 8 (part of SAP NetWeaver 7.02 SP 8)PI BASIS 730 SP 3 (part of SAP NetWeaver 7.30 SP 3)PI BASIS 731 SP 1 (part of SAP NetWeaver 7.03 SP 1 and 7.31 SP 1) ERP 6.0 SP 20ERP 6.0 EhP 2 SP 10ERP 6.0 EhP 3 SP 09ERP 6.0 EhP 4 SP 10ERP 6.0 EhP 5 SP 05See SAP Note 1521883 - ODP Data Replication API for further details.ConsumerSubscriber /Consumer Recommended starting release with BW 7.40 2014 SAP SE or an SAP affiliate company. All rights reserved.Public25
ODP enabled ExtractorsEnablementEnable Extractors for ODP framework The SAP Note Releasing ERP Extractors for ODP API together with SAP Note 1558737 - Data Sourcesreleased for ODP data replication API describes which Data Sources have been released for usage with ODPData Replication API: Examples: 0FI GL 50‚ 0HR PA EC 03‚ 0MATERIAL ATTR, 2LIS 11 V ITM, '0BPARTNER ATTR‚'0CO OM CCA 1‚ 0EC CS 3‚ 0CO PC ACT 1 To use the ODP data replication API for any generic DataSource (extraction methods view extraction or domainextraction) you need to implement SAP Note 1585204. Single Extractors can be released with RODPS OS EXPOSE 2014 SAP SE or an SAP affiliate company. All rights reserved.Public26
Example: One queue multiple consumersSubscriber /ConsumerProviderExample showing the flexibility of ODQ Automated handling of one queue formultiple subscribers withoutmultiplying the dataTarget BWOperationalDataProvisioningSAP ERP ExtractorsODQTarget BW Example: one ERP Extractor, manyBW Subscribers (global/localinstances) Retention period until all subscriberreceived the data successfullyTarget BW 2014 SAP SE or an SAP affiliate company. All rights reserved.Public27
Implementation QuestionsSubscriber /ConsumerProviderSource ERPODQODP DataSource2LIS 11 V ITMCan ODP be deployed in parallel withthe traditional delta queueapproach?Yes it is possible, but multiplies thedata.Should we change to ODP basedextraction with all existingextractors?2LIS 11 V ITMTarget BWDelta Queue 2014 SAP SE or an SAP affiliate company. All rights reserved.SAP ERP DataSource2LIS 11 V ITMNo, but consider ODP as framework forall your future implementations of newdata flows into you BW system for ECCand SLT extraction.Public28
Reproduction of BW Extractors with SLT‘s View functionalityView-based SLT delta-transfer technology (with SP06):BW Extractors can be rebuilt with Views, if the following prerequisites are fulfilled: Views need to be built which need to contain all relevant tables the view relations as well as the primary key of the view need to be designed there must be a hierarchy with one main table only this main table is triggered/recorded (means the line item tables attached to the main table are not triggered) Target is 1 transparent table (there is no data split after SLT transfer e.g. into two or more tables)The capability is shipped as a framework; content is not part of the shipment!Benefits compared to standard BW Extractors:- Can go across clients, reducing number of running processes and setup efforts for each client- enhances tables with delta capabilities, thus streamlines the dataflow from full to delta for transferred data. 2014 SAP SE or an SAP affiliate
Providing BW Business Data for Big Data Scenarios SAP HANA Spatial Analytics SAP HANA Sentiment Intelligence RDS . DMM300 - SAP Business Warehouse Data Provisioning from SAP and Non-SAP Sources Introduction – Data Provisioning Aspects regarding SAP Business Warehouse Traditional understanding Data movement into one central repository (ETL processing, Replication) for good
