Transcription
fSAP NetWeaver Process Integration (PI)Performance Check Guide:Analyzing Performance Problems and Possible SolutionStrategiesSAP NetWeaver PI 7.1xSAP NetWeaver PI 7.3xSAP NetWeaver PI 7.4Document Version 3.0March 2014
SAP NETWEAVER 7.1 AND HIGHERPERFORMANCE CHECK GUIDEDOCUMENT HISTORYBefore you start planning, make sure you have the latest version of this document. You can find the link tothe latest version via SAP Note 894509 – PI Performance Check.The following table provides an overview on the most important changes to this document.VersionDateDescription3.0March 2014Reason for new version: Include latest improvements in PI 7.31 andPI 7.4 and enhanced tuning options for Java only runtime. Manysections were updated. Major changes are: Additional symptom (blacklisting) for qRFC queues in READYstatus Long processing time for Lean Message Search IDoc posting configuration on receiver side Tuning of SOAP sender adapter More information about IDOC AAE adapter tuning Packaging for Java IDoc and Java Proxy adapter New Adapter Framework Scheduler for polling adapters new performance monitor for Adapter Engine Enhancements of the maxReceiver parameter for individualinterfaces Include latest information for staging and logging (restructured tobe part of Java only chapter) FCA Thread tuning for incoming HTTP requests Enhancements in the Proxy framework on sender/receiver ERPs Large message queues on PI Adapter Engine Generic J2EE database monitoring in NWA Appendix section: XPI Inspector2.0October 2011Reason for new version: Adaption to PI 7.3 Added document history Added options and description about AEX Added new adapters / updated existing adapter parallelization Description of additional monitors (Java performance monitorand new ccBPM monitor) General review of entire document with corrections More details and performance measurements for Java Only2
SAP NETWEAVER 7.1 AND HIGHERPERFORMANCE CHECK GUIDE(ICO) scenariosAdditional new chapters: Prevent blocking of EO queues Avoid uneven backlogs with queue balancing Reduce the number of EOIO queues Adapter Framework Scheduler Avoid blocking of Java only scenarios J2EE HTTP load balancing Persistence of Audit Log information in PI 7.10 and higher Logging / Staging on the AAE (PI 7.3 and higher)1.1December 2009Reason for new version: Adaption to PI 7.1 General review of entire document with correctionsAdditional new chapters: Tuning the IDoc adapter Message Prioritization on the ABAP stack Prioritization in the Messaging System Avoid blocking caused by single slow/hanging receiver interface Performance of Module Processing Advanced Adapter Engine / Integrated Configuration ABAP Proxy system tuning Wily Transaction Trace1.0October 2007Initial document version released for XI 3.0 and PI 7.03
SAP NETWEAVER 7.1 AND HIGHERPERFORMANCE CHECK GUIDETABLE OF CONTENTS1INTRODUCTION . 62WORKING WITH THIS DOCUMENT . 1033.13.23.2.13.3DETERMINING THE BOTTLENECK . 12Integration Engine Processing Time . 12Adapter Engine Processing Time . 13Adapter Engine Performance monitor in PI 7.31 and higher . 15Processing Time in the Business Process Engine . 164ANALYZING THE INTEGRATION ENGINE . 204.1Work Process Overview (SM50/SM66) . 214.2qRFC Resources (SARFC) . 224.3Parallelization of PI qRFC Queues . 234.4Analyzing the runtime of PI pipeline steps . 294.4.1Long Processing Times for “PLSRV RECEIVER DETERMINATION” /PLSRV INTERFACE DETERMINATION. 304.4.2Long Processing Times for “PLSRV MAPPING REQUEST” . 304.4.3Long Processing Times for “PLSRV CALL ADAPTER” . 334.4.4Long Processing Times for “DB ENTRY QUEUEING” . 344.4.5Long Processing Times for “DB SPLITTER QUEUEING” . 344.4.6Long Processing Times for “LMS EXTRACTION” . 364.4.7Other step performed in the ABAP pipeline. 364.5PI Message Packaging for Integration Engine. 374.6Prevent blocking of EO queues . 384.7Avoid uneven backlogs with queue balancing . 384.8Reduce the number of parallel EOIO queues . 404.9Tuning the ABAP IDoc Adapter . 414.9.1ABAP basis tuning . 414.9.2Packaging on sender and receiver side . 424.9.3Configuration of IDoc posting on receiver side . 424.10Message Prioritization on the ABAP Stack . 4455.15.25.35.45.55.6ANALYZING THE BUSINESS PROCESS ENGINE (BPE) . 45Work Process Overview . 45Duration of Integration Process Steps . 45Advanced Analysis: Load Created by the Business Process Engine (ST03N) . 47Database Reorganization . 47Queuing and ccBPM (or increasing Parallelism for ccBPM Processes) . 48Message Packaging in BPE . .2.36.2.46.2.56.36.4ANALYZING THE (ADVANCED) ADAPTER ENGINE . 50Adapter Performance Problem . 51Adapter Parallelism. 51Sender Adapter . 54Receiver Adapter . 55IDoc AAE adapter tuning . 56Packaging for Proxy (SOAP in XI 3.0 protocol) and Java IDoc adapter . 56Adapter Framework Scheduler . 58Messaging System Bottleneck . 60Messaging System in between AFW Sender Adapter and Integration Server (Outbound) . 63Messaging System in Between Integration Server and AFW Receiver Adapter (Inbound) . 64Interface Prioritization in the Messaging System. 64Avoid Blocking Caused by Single Slow/Hanging Receiver Interface . 65Overhead based on interface pattern being used . 67Performance of Module Processing . 68Java only scenarios / Integrated Configuration objects . 694
SAP NETWEAVER 7.1 AND HIGHERPERFORMANCE CHECK 4General performance gain when using Java only scenarios . 69Message Flow of Java only scenarios . 71Avoid blocking of Java only scenarios . 73Logging / Staging on the AAE (PI 7.3 and higher) . 73J2EE HTTP load balancing . 75J2EE Engine Bottleneck. 76Java Memory . 76Java System and Application Threads . 80FCA Server Threads . 83Switch Off VMC . 8477.1ABAP PROXY SYSTEM TUNING . 85New enhancements in Proxy queuing . 8688.18.2MESSAGE SIZE AS SOURCE OF PERFORMANCE PROBLEMS . 87Large message queues on PI ABAP . 88Large message queues on PI Adapter Engine . 8899.19.29.39.3.19.3.29.3.39.3.49.3.59.4GENERAL HARDWARE BOTTLENECK . 90Monitoring CPU Capacity. 90Monitoring Memory and Paging Activity . 91Monitoring the Database . 91Generic J2EE database monitoring in NWA. 92Monitoring Database (Oracle) . 93Monitoring Database (MS SQL) . 94Monitoring Database (DB2) . 95Monitoring Database (MaxDB / SAP DB) . 97Monitoring Database Tables . 991010.110.210.310.3.1TRACES, LOGS, AND MONITORING DECREASING PERFORMANCE . 101Integration Engine . 101Business Process Engine . 102Adapter Framework . 102Persistence of Audit Log information in PI 7.10 and higher . 10311ERRORS AS SOURCE OF PERFORMANCE PROBLEMS . 104APPENDIX A . 105A.1Wily Introscope Transaction Trace . 105A.2XPI inspector for troubleshooting and performance analysis . 1065
SAP NETWEAVER 7.1 AND HIGHER1PERFORMANCE CHECK GUIDEINTRODUCTIONSAP NetWeaver Process Integration (PI) consists of the following functional components: Integration Builder(including Enterprise Service Repository (ESR), Service Registry (SR) and Integration Directory), IntegrationServer (including Integration Engine, Business Process Engine and Adapter Engine), Runtime Workbench(RWB), and System Landscape Directory (SLD). The SLD, in contrast to the other components, may notnecessarily be part of the PI system in your system landscape because it is used by several SAP NetWeaverproducts (however it will be accessed by the PI system regularly). Additional components in your PIlandscape might be the Partner Connectivity Kit (PCK), a J2SE Adapter Engine, one or several non-centralAdvanced Adapter Engines or an Advanced Adapter Engine Extended. An overview is given in the graphicbelow. The communication and accessibility of these components can be checked using the PI ReadinessCheck (SAP Note 817920).Processing at runtime is carried out by the Integration Server (IS) with the aid of the SLD. The IS in thisgraphic stands for the Web Application Server 7.0 or higher. In the classic environment PI is installed as adouble stack: an ABAP and a Java stack (J2EE Engine). The ABAP stack is responsible for pipelineprocessing in the Integration Engine (Receiver Determination, Interface Determination, and so on) and theprocessing of integration processes in the Business Process Engine. Every message has to pass throughthe ABAP stack. The Java stack executes the mapping of messages (with the exception of ABAP mappings)and hosts the Adapter Framework (AFW). The Adapter Framework contains all XI adapters except the plainHTTP, WSRM, and IDoc adapters.6
SAP NETWEAVER 7.1 AND HIGHERPERFORMANCE CHECK GUIDEWith 7.1 the so called Advanced Adapter Engine (AAE) was introduced. The Adapter Engine was enhancedto provide additional routing and mapping call functionality so that it is possible to process messages locally.This means that a message that is handled by a sender and receiver adapter based on J2EE does not needto be forwarded to the ABAP Integration Engine. This saves a lot of internal communication, reduces theresponse time, and significantly increases the overall throughput. The deployment options and the messageflow for 7.1 based systems and higher are shown below: Currently not all the functionalities available in thePI ABAP stack are available on the AAE but each new PI release closes the gap further.In SAP PI 7.3 and higher the Adapter Engine Extended (AEX) was introduced. In addition to the AAE theAdapter Engine Extended also allows local configuration of the PI objects. The AEX can therefore be seenas a complete PI installation running on Java only. From the runtime perspective no major differences can beseen compared to the AAE and therefore no differentiation is made in this guide.7
SAP NETWEAVER 7.1 AND HIGHERPERFORMANCE CHECK GUIDELooking at the different involved components we get a first impression of where a performance problemmight be located: In the Integration Engine itself, in the Business Process Engine, or in one of the AdvancedAdapter Engines (central, non-central, plain, PCK). Of course, a performance problem could also occuranywhere in between, for example, in the network or around a firewall. Note that there is a separationbetween a performance issue “in PI” and a performance issue in any attached systems: If viewed “throughthe eyes” of a message, (that is, from the point of view of the message flow), then the PI system technicallystarts as soon as the message enters an adapter or (if HTTP communication is used) as soon as themessage enters the pipeline of the Integration Engine. Any delay prior to this must be handled by thesending system. The PI system technically ends as soon as the message reaches the target system, forexample, a given receiver adapter (or in case of HTTP communication, the pipeline of the Integration Server)has received the success message of the receiver. Any delay after this point in time must be analyzed in thereceiving system.There are generally two types of performance problems. The first is a more general statement that the PIsystem is slow and has a low performance level, or does not reach the expected throughput. The second istypically connected to a specific interface failing to meet the business expectation with regard to theprocessing time. The layout of this check is based on the latter: First you should try to determine thecomponent that is responsible for the long processing time or the component that needs the highest absolutetime for processing. Once this has been clarified, there is a set of transactions that will help you to analyzethe origin of the performance problem.If the recommendations given in this guide are not sufficient, SAP can help you to optimize the service viaSAP consulting or SAP MaxAttention services. SAP might also handle smaller problems, restricted to aspecific interface, if you describe your problem in a SAP customer incident. Support offerings like “SAPGoing Live Analysis (GA)” and “SAP Going Live Verification (GV)” can be used to ensure that the mainsystem parameters are configured according to SAP best practice. You can order any type of service usingSAP Service Marketplace or your local SAP contacts.If you have already worked with the PI Admin Check (SAP Note 884865) you will recognize some of thetransactions. This is because SAP considers the regular checking of the performance to be an importantadministrational task. However, this check tries to show the methodology to approach performance problemsto its reader. Also, it offers the most common reasons for performance problems and links to possible followup actions but does not refer to any regular administrational transactions.8
SAP NETWEAVER 7.1 AND HIGHERPERFORMANCE CHECK GUI
SAP NETWEAVER 7.1 AND HIGHER PERFORMANCE CHECK GUIDE 3 (ICO) scenarios Additional new chapters: Prevent blocking of EO queues Avoid uneven backlogs with queue balancing Reduce the number of EOIO queues Adapter Framework Scheduler Avoid blocking of Java only scenarios J2EE HTTP load balancing Persistence of Audit Log information in PI 7.10 and higher
