WIXLIB

Benchmark TestEnvironmentAccenture’s engineers performed thetesting, using their own data and theAzure platform. This means that thetesting wasn’t performed in a lab butrather in the actual cloud environment acarrier could use in deploying a privatecloud ALIP solution.Figure 1: The ALIP EnvironmentApplication ServerDS14 - 16CPU/112GB RAMCentOS 6.6Jboss EAP 6.4ALIP Process ServerALIP Process ServerDS14 - 16CPUJ/112GB RAMCentOS 6.6Java 1.7DS14 - 16CPU/112GB RAMCentOS 6.6Java 1.7Database ServerGS5 - 32CPU/448GB RAMWindows Server 2012 R2SQL Server 2014The ALIP environment, as shown in Figure1, was created using four Azure virtualmachine (VM) nodes. A fifth Azure VMwas used for executing test scripts. Thefour ALIP virtual machines were selectedfrom Azure template offerings, as shownin Table 1.Tests simulating concurrent user activityinvolved interactions between theapplication server, process servers, anddatabase. Batch testing doesn’t involvethe application server, as batch processingis handled by the process servers and thedatabase.20x1TB Premium Storage DataTable 1: Virtual Machine DescriptionsVirtual MachinesVirtual MachinesDetailsDatabase ServerAzure GS5 template for 1 VM with: 32 CPUs 448 GB memory 20 1-terabyte drives for storage 6 1-terabyte drives for log storage 5 1-terabyte drives for backups Windows Server 2012 R2 operating system Microsoft SQL Server 2014Database 15 terabytes total data 1,500 tables 5.6 billion rows in largest table (Premium History)Application ServerAzure DS14 template for 1 VM with: 16 CPUs 112 GB memory Linux CentOS 6.6 operating systemTest SoftwareSoftware that was used to facilitatetesting included: Load Generation. The load testingfeature of Microsoft Visual StudioUltimate was used for load generation,maintaining simulated users, the testrunner, and for collecting results,working in conjunction with a testcontroller application. Batch Performance Test Tool. The ALIPPerformance Team created a batchperformance test tool. The tool, writtenin the Groovy scripting language, wasdesigned to run and time a series ofbatch jobs. The tool also collected a setof output data providing further detailsabout the run of the job.6x1TB Premium Storage Logs JBoss Enterprise Application Platform 6.4Note: One installation of JBoss was used with two JVMs to split the3,000 users for load-balancing.ALIP ProcessServersAzure DS14 template for 2 VMs, each with: 16 CPUs 112 GB memory Linux CentOS 6.6 operating system Java 1.7Note: Each VM ran six process servers, with each set to process up to10 threads of work, for a total of 12 process servers with the ability tohandle 120 simultaneous threads of work.3

Test DatabaseThe 15-terabyte database used in thetesting included contracts from thefollowing product types: Term Whole Life Variable Universal Life Deferred Annuity Immediate AnnuityThe contract data was entered so thatthe data was spread across several tables,including application, policy, customer,general ledger, and requirements. Table 2provides a partial list of approximate tablerow counts from the test runs.Table 2: Partial Listing of Database TablesTest Database Tables (Partial Listing)Table TypeMain Table RowsNotesApplication41 MillionThere are multiple rows in this table for each actualapplication.Clients19 MillionIn this context, client refers to any person ororganization involved with ALIP or a policy in anyway. All ALIP users have a row, all policy owners,beneficiaries, agents, etc.Policy Roles35 MillionAll people associated with a policy, including owner,joint owner, insured (even if the same as owner, thistable must have a unique row by role), agent, etc.Transactions247 MillionTransactions at a policy level.Billing Records61 MillionNumber of bills.Requirements53 MillionApplication, client, policy or transaction level requirements which must be met in order to proceed.General Ledger85 MillionGeneral ledger entries.Tasks134 MillionTasks requiring the attention of ALIP users.CashValue Quote309 MillionCurrent and historical records of the financial valuesof the policies.GenericFeature Values1.7 BillionStores policy’s running values needed for productcalculation for every transaction.PremiumHistory Values5.6 BillionThis is the largest table. It stores history about all thepremiums paid in to policy.4

Online Load &Scalability TestFive separate tests, referred to as webtests, were created, and then loadedinto Visual Studio to be executedsimultaneously as a load test. Scalabilitywas measured by performing all five webtests using the loads generated by a singleuser, and then repeating the tests withloads generated by 3,000 users.Table 3: Workflow for the Term Life and Whole Life Insurance Application TestsLife Insurance Application TestTestDescriptionTerm Insuranceand Whole Life usedthe same procedureOpen Term Life (or Whole Life) Insurance Application – Visual Pages Welcome Screen In the new application workflow:-Methodology for OnlineLoad & Scalability TestThroughout testing, the goal was toreplicate real-life workloads as much aspossible—even more so. In fact, the actualworkloads tested exceeded what mostTier 1 carriers would likely see. To providea cross section of real-life workloads,three types of application workflows wereused, as well as two common forms ofmaintenance. The scenarios used for thefive web tests were: Term Life Insurance Application Test Whole Life Insurance Application Test Immediate Annuity Application Test Policy Maintenance Test Financial Maintenance Test LoginProduct Type SelectionProduct SelectionInsured and Owner InformationPersonal and Medical HistoryPolicy and Premium InformationReplacement InformationApplication Summary- Forms Summary Application List LogoutTable 4: Workflow for the Immediate Annuity Application TestImmediate Annuity Application TestTestDescriptionImmediate AnnuityApplicationOpen Immediate Annuity Application – Visual Pages Login Welcome Screen In the new application workflow:A closer look at what was involved in eachtest is described next.Term Life and Whole Life InsuranceApplication TestsWorkflow for the Term Life InsuranceApplication test included opening webpages, accepting inputs, replicating theorder in which an agent would enter data.The same workflow was used for theWhole Life Insurance Application Test.Workflow for both is shown in Table 3.Immediate Annuity Application TestWorkflow for the Immediate AnnuityApplication test included opening webpages, accepting inputs, replicating theorder in which an agent would enter data.The workflow is described in Table 4.-Product Type SelectionApplication DetailsProduct SelectionAnnuitant InformationJoint Owner/Annuitant InformationJoint Owner ConfirmationRule Group PackagePremium InformationAdditional Contract InformationFinancial stream summary information pageAdd a StreamStream InformationFund InformationFinancial stream summary information page (Return here afteradding details)- Replacement Information- Application Summary- Forms Summary Application List Logout5

For all three application entry testsrandomized data was used for key fieldsin the questionnaire such as names andSocial Security numbers. The applicationentry tests were done primarily within theapplication server VM and the databaseuntil the very end of the applicationentry when a request is sent to theprocess server, via the message queue, forunderwriting processing.Table 5: Workflow for the Policy Maintenance TestPolicy Maintenance TestTestDescriptionModifying aCustomer AddressOpen Modify Customer Address – Visual PagesPolicy Maintenance (Modify CustomerAddress) TestPolicy maintenance testing was based uponthe common task of modifying a customeraddress. The policy maintenance workflowis shown in Table 5.The financial maintenance test is doneentirely within the application server VMand database.LoginWelcome ScreenContract SearchContract Search ResultsContract Summary PageContract Relationships PageCustomer Profile PageCustomer Address Management PageIn the address change workflow:- Address Edit screen- Address summary Customer Address Management Page (Return here after workflow)The policy maintenance test is done entirelywithin the application server VM anddatabase.Financial Maintenance (Create andApply Suspense) TestFinancial maintenance testing was basedupon the common task of create andapply suspense—the process of receivinga payment and manually matching thepayment to the contract. In this test thepayment was applied immediately, creatinga billing premium payment transaction. Thefinancial maintenance workflow for the testis shown in Table 6. LogoffTable 6: Workflow of the Financial Maintenance TestFinancial Maintenance TestTestDescriptionCreate andApply SuspenseOpen Create and Apply Suspense – Visual Pages Login Welcome Screen Suspense List In the Suspense workflow:- Suspense Information Entry- Suspense Summary Suspense List (Return here after workflow) Logoff6

Configurationsfor Online Load& Scalability TestThe five web test scenarios describedabove were combined into a Visual Studioload test. The load test was configured forthe number of users assigned to each webtest, the number of times each user shouldperform the test per hour and the time ofthe test itself.Two versions of load tests were created: Single User Baseline. In the single userbaseline, each load test was run onetime sequentially by one user. The resultswere calculated in the same way thatthey would be if a large load had beenplaced on the system. This allowed for acomparison between how ALIP respondswhen only a single user is performing atask versus when a large load is placedon the system. Multiple User Testing. The multiple usertesting simulated 3,000 simultaneoususers performing the individual testscenarios for 1 hour.To simulate real-life workplace usage, eachof the 3,000 users generated one action,drawn from the five web tests, duringthe hour of testing. One simulated personentered data for a new Term Life insurancepolicy, while another modified a customeraddress. The workloads from all 3,000 userswere executed within the hour.Configuration for the 3,000 userassignments, as shown in Table 7, were for200 users to enter Whole Life insuranceapplications, while 400 entered data forTerm Life applications and 400 for animmediate Annuity application, while 1,000performed a create and apply suspense, and1,000 modified a customer address. Notethere is slight variation in the requestedpace per hour and the actual total testsin the hour, as a function of how thetest engine randomized the total flow ofassignments. For example, 200 Whole Lifeapplications were scheduled to be entered,and 205 were actually entered, as shown inTable 7. So the actual tests performed areactually a bit higher.Test FindingsThe online load testing showed impressivescalability, as there was very littledifference between page loading timeswhen comparing the single-user results tothose of the 3,000-user test. As shown inTable 8, the average page serving time fora single user was 0.21 seconds difference,compared to the page load time with3,000 users of 0.25 seconds—a mere 0.04seconds difference. A total of 228,615pages were served up during the 1 hourtest, processing 1,764,999 individualrequests.Testing also found that the percent of pagesserved in less than 1 second remained at93% when scaling from one user to 3,000,as shown in Table 8. The single page thatexceeded 2 seconds after scaling to 3,000users only exceeded the threshold by 0.05seconds, loading in at 2.05.When looking at CPU usage, we also findenterprise-grade scalability. While CPU usagewas negligible for a single user, increasingthe load by a factor of 3,000, only raisedCPU utilization to an average of about 45%.The ALIP architecture and Azure deploymentmeans that at any point you could reduceCPU use further by simply adding anotherapplication performance node.Table 7: Total DistributionTesting Distribution for Online Load & Scalability TestingTestUsersRequested PaceTotal Tests in 1 HourTerm Life Application Entry4001 per hour424Whole Life Application Entry2001 per hour205Immediate Annuity Application Entry 4001 per hour352Suspense Create and Apply10001 per hour1,056Modify Customer Address10001 per hour1,050Table 8: Page Serving Time Test ResultsTest Results for Online Load & Scalability Testing Showing Page Loading TimesCriteriaSingle User3,000 UsersNumber of unique pages hit2929Average page time overall (in seconds)0.210.25Pages served in less than 1 second(number & percentage)27/93%27/93%Pages served in less than 1 to 2 seconds(number & percentage)2/7%1/3.5%Pages served in more than 2 seconds0/0%1/3.5%Note: The 1 page that exceeded 2 seconds had a response time of 2.05 seconds7

Batch TestTest FindingsAccenture also tested performanceof batch jobs. ALIP batch jobs requirespecific data to exist at the contract level.As such, the number of contracts for eachtest varies slightly, ranging in these testsfrom approximately 75,000 to 100,000contracts.The batch testing, with its onlinetransaction processing, demonstratedthat ALIP deployed on the Azure platformprovides Tier 1 levels of processing power.As shown in Table 10, the billing batch jobprocessed 100,000 contracts at a rate of50 contracts per second, completing thebatch in just 33 minutes. The complexcycle batches were handled at a rate of18 contracts per second, completing some75,000 contracts in just under 69 minutes.The jobs used for the batch testing loadsare shown in Table 9.CPU usage for the cycle test, as shown inChart 1, was about 50% for the databaseserver and about half that for the twoALIP process servers. The initial spike forthe process servers reflects loading thecontract data, along with efficiency gainsafter the spike from caching.Table 9: Description of Batch Test WorkloadsBatch Test WorkloadsBatch Job NameDescriptionBillingDetermines which contracts need to be billed for upcoming premiumand generates a billing record for further processing. Testing used anelectronic fund transfer (EFT) mode of payment which also generates apending Billing Premium Payment transaction.Premium Due Date(PDD)Analyzes a set of contracts to determine when a premium payment isnext due on the contract, as well as payment details, including amountdue. The PDD batch then creates a pending transaction for the premiumdue date.CycleThe cycle job can include a wide variety of operations on contracts. Fortesting, 3 common cycle activities were used: Billing. Similar to the billing batch job described above. For testingonly EFT transactions were used. Cash Value Quote. Determines the current cash value of theinsurance or annuity contract. Transaction Processing. Process any pending transactions on thecontracts, such as premium due date and billing premium paymenttransactions. The Billing and PDD jobs perform the necessarycalculations and stores the data, but the actual changes to thecontract are applied during cycle.Table 10: Results of Batch TestingBatch Test ResultsJob NameContractsProcessedContractsper SecondTime inSecondsTime 9.375161.3086.02Cycle74,99518.164129.8868.838

Accenture testers performed a final batchtest, using a single component of thecycle batch—the calculation-heavy job ofgenerating cash value quotes—for half amillion contracts. The cash value quote wasrun against 499,683 contracts at a rate of68.24 per second, to accomplish the taskin just over two hours (122.05 minutes).CPU usage for this test is shown in Chart2, with the database showing about 50%utilization and the two ALIP process serversrunning at about 20%.Chart 1: Cycle test CPU utilization for database (red) and policy servers (blue and green)Test Team ObservationsThe Accenture testing team was impressedwith what the testing revealed aboutdeploying ALIP on the Azure platform.“Going into the tests, our assumptionwas that the cloud-based version of ALIPwould be slower than what our tests hadpreviously found using our own physicalservers,” one engineer said. “What wefound instead was that ALIP on Azure wasactually faster—with heavier workloads.”Chart 2: Cash value quote CPU utilization for database (red) and process servers (blue and green)Another engineer underscored that thetesting was performed with very heavyloads. “We purposely chose the mostdemanding jobs to create very heavyworkloads,” the engineer said. “We didn’tcherry pick. In fact, we worked withvolumes that exceed what many Tier 1insurers would likely face in a single night.”SummaryBenchmark testing of ALIP on Azure,supported by Microsoft SQL Server 2014data management software and runningon the Windows Server 2012 R2 operatingsystem, found that ALIP provided theperformance and scalability to meetthe needs of Tier 1 insurance carriers.Scalability was demonstrated by the easewith which workloads were scaled, withaverage page serving time of 0.21 secondsfor a single user rising by only 0.04 to 0.25seconds when scaled to 3,000 users.Testing also showed no change fromthe 93% of pages delivered sub-secondwhen scaled from 1 to 3,000 users. Highavailability can be achieved through ALIP’srobust architecture and the ease withwhich work can be transferred from oneAzure node to another, on the fly.9

About MicrosoftFounded in 1975, Microsoft (Nasdaq“MSFT”) is the worldwide leader insoftware, services, and solutions thathelp people and businesses realize theirfull potential.For more information about Microsoftproducts and services, visitwww.microsoft.com.About AccentureAccenture is a leading global professionalservices company, providing a broadrange of services and solutions instrategy, consulting, digital, technologyand operations. Combining unmatchedexperience and specialized skills acrossmore than 40 industries and all businessfunctions—underpinned by the world’slargest delivery network—Accentureworks at the intersection of business andtechnology to help clients improve theirperformance and create sustainable valuefor their stakeholders. With more than358,000 people serving clients in more than120 countries, Accenture drives innovationto improve the way the world works andlives. Visit us at www.accenture.comCopyright 2016 AccentureAll rights reserved.Accenture, its logo, andHigh Performance Deliveredare trademarks of Accenture.This document is produced by consultants atAccenture as general guidance. It is not intendedto provide specific advice on your circumstances.Copyright 2016 Microsoft.All rights reserved.This document is provided “as-is.” Information and views expressed in this document, including URLand other Internet Web site references, may change without notice. This document does not provideyou with any legal rights to any intellectual property in any Microsoft product. You may copy and usethis document for your internal, reference purposes.

Accenture's successful benchmark testing of Accenture Life Insurance & Annuity Platform (ALIP) for Tier 1 life and annuity carrier workloads on Microsoft Azure. Accenture, a global management consulting, technology services and outsourcing company, with more than 358,000 people serving clients in more than 120 countries, is dedicated to