WIXLIB

International Journal of Scientific & Engineering Research, Volume 7, Issue 12, December-2016ISSN 2229-5518327Performance Evaluation of JMeter,LoadComplete and WAPTReenu BhatiaResearch ScholarDepartment of Computer Science Himachal Pradesh University Shimla - 171005Email : bhatia.renu9@gmail.comAnita GanpatiAssociate ProfessorDepartment of Computer Science Himachal Pradesh University Shimla - 171005Email : ----------------------------------------------Load testing is used to monitor the changes in system performance with increase in load to thesystem. Load testing is a non functional testing which is used to understand the system behaviorunder a specific expected load. At present there is a huge shift towards web applications and largenumbers of tools are available in the market for web application performance analysis. In thispaper three load testing tools namely Apache JMeter, LoadComplete and WAPT are compared.The main focus of the study is to discuss these load testing tools and compare them on the basis ofkey parameter called response time. The response time of these tools is compared by varyingnumber of concurrent users. The primary objective is to study these load testing tools and identifywhich one of them is better and more efficient .In this study it is concluded that in term of responsetime WAPT has better performance than that of Apache JMeter and LoadComplete.IJSERKeywords – Apache JMeter, LoadComplete, Load Testing Tools, Performance Testing, ResponseTime, WAPT.1. IntroductionTesting, Spike Testing, Endurance Testing,Volume Testing and Scalability Testing.Software testing is an important phase ofsoftware development life cycle. Any softwareunder development go through various types oftesting to identify and quantify its quality.Testing is a process used for disclosing defectsin software and establishing that software hasgained a specified degree of quality with respectto selected attributes [2].Today there is a huge shift towards webapplications; therefore it is necessary todetermine the performance of web applications.Performance of web applications can bedetermined in terms of Availability, ResponseTime, Throughput, Utilization and Latency [10].To determine the performance of webapplications, various types of performancetesting can be done i. e. Load Testing, StressLoad testing is used to determine the changes insystem performance with increase in load to thesystem. The load can be determined in terms ofnumber of concurrent users. Load testing is anon functional testing which is used tounderstand the system behavior under a specificexpected load. It is conducted to determine thesystem behavior under normal and peak loadconditions [12]. Load generator is used tosimulate real life user load for the targetapplication.In this paper, section 1 gives the introductionabout load testing and section 2 describes theliterature survey related to the study. Section 3describes the introduction about the selectedtools for the study. Section 4 describes theIJSER 2016http://www.ijser.org

International Journal of Scientific & Engineering Research, Volume 7, Issue 12, December-2016ISSN 2229-5518comparative analysis of selected testing toolsand section 5 describes the conclusion and futurescope of the work.2. Literature SurveyMonika Sharma et al. [11] gave a comparativestudy of JMeter, HP LoadRunner, WebLOADand Grinder on the basis of parameters likeServer Monitoring, Unlimited Load generation,ease of use, cost, etc. After comparison it isconcluded that JMeter is best tool as it is free,having great load generation and easy userinterface. Manju Kaushik and Pratibha Fageria[6] conducted a comparative study onperformance analysis of Neoload, WAPT,LoadUI on the basis of parameters likethroughput, response time, number of hit pages,error rate, memory and CPU utilization etc.Rigzin Angmo and Monika Sharma [1] gave aPerformance Evaluation of Web BasedAutomation Testing Tools namely seleniumwebdriver and watir webdriver. Here theperformance of these testing tools is evaluatedand compared, and concluded that watirwebddriver is suitable under specific situation,but selenium webdriver is better choice invarious conditions like using domain specificlanguage. Cheng-hui Huang, and Huo Yan Chen[4] gave a Tool to Support Automated Testingfor Web Application Scenario named WASATT(Web Application Scenario Automated TestingTool), the tool support the automated testing forscenario of web-based applications. VandanaChandel et al. [3] has done a comparative studyof testing tools: Apache JMeter and LoadRunner which compare these tools based on thecriteria such as performance, speed, throughputand efficiency and concluded that JMeter isbetter tool to go forward with. Harpreet Kaurand Gagan Gupta [5] conducted a comparativestudy of automated testing Tools: Selenium,Quick Test Professional and Testcomplete onthe basis of their usability and effectiveness andconcluded that one can select a testing toolbased on the type of application need to betested, budget and the efficiency required. VinitaMalik and Mamta Gahlan [9] has given acomparative study of automated web testingtools of automated testing namely Quick TestProfessional, Selenium, Watir and Sahi based on328the criteria such as efforts involved withgenerating test scripts, capability to play backthe scripts, result reports, speed and cost andconcluded that QTP is the best tool among themall. Rifa Nizam Khan and Shobhit Gupta [8] hasgiven a comparative study of automated testingtools: Rational Functional Tester, Quick TestProfessional, Silk Test and Loadrunner anddetermine their usability and efficiency andconcluded QTP is a good tool. Dipika Kelkarand Kavita Kandalgaonkar [7] has given ananalysis and comparison of performance testingtools namely LoadRunner and JMeter anddetermine their accuracy of responses andrecommend going ahead with HP LoadRunneras it is very stable and robust.3. Overview of Load Testing ToolsThere are number of load testing tools availablein the market. Load testing tools help todetermine the performance of web applicationunder heavy load and quantify the elementsresponsible for performance degradation. Theysimulate heavy load in terms of number ofconcurrent users and thus help to analyze thesystem performance under different loadconditions. In this paper, three load testing toolsareselectednamelyApacheJMeter,LoadComplete and WAPT.IJSERApache JMeter an open source, a 100% pureJAVA application designed to load test andfunctional behavior [13]. It is a cross – platformtool developed by Apache Software Foundation.JMeter simulates the number of users sendingrequest to the target server, return statistics andshow the performance of target server in theform of graph, table etc [16].LoadComplete is used for load, stress,scalability testing of websites and webapplications [14]. LoadComplete is a freeforever load testing tool developed bySmartBear Software used for on demand loadgeneration, uses record and replay to record aload test to test a wide variety of web applicationand easily monitor server performance.WAPT is Web Application Performance Testingtool used for load, stress, performance testing ofweb applications [15]. WAPT supportsIJSER 2016http://www.ijser.org

International Journal of Scientific & Engineering Research, Volume 7, Issue 12, December-2016ISSN 2229-5518distributed load generation used for server anddatabase performance monitoring. WAPTsimulates real life condition as accurate aspossible and gives detailed test report withgraphical representation.5LatestRelease6License andPricing7BrowserSupport8Requirements4. Comparative Analysis of Load TestingToolsThis section represents a comparative analysis ofthe selected tools along with their observedresults. The tests were conducted at the sameinstant of time at same network speed. For thisstudy we use the current version of JMeter thatis 2.13 r1665067, LoadComplete 4 and WAPT9.0. Table I represents basic information aboutselected load testing tools. Whereas in Table IIthe performance of tools was evaluated on thebasis of critical parameter called response time.ResponsetimeofApacheJMeter,LoadComplete and WAPT is measured byvarying the number of concurrent users from 1virtual user to 10 virtual users and 20 virtualusers.14-03201503-03201617-082015OpenSource freeToolFreeforever upto 50 VUsMultiBrowserMultiBrowser700USD 300USDMaintenancefee peryearMultiBrowserJava6 MicrosoftInternetInformation Services(IIS) 7.0or later,IJSER.NETFramework 3.5 orlater,Table I: Basic Information of Load sPlatformLoadCompleteWindows7 or crosoftDataAccess (ifyou needto workwith thedatabasefiles).Windows14-102003IJSER 2016http://www.ijser.org50-500MBdiskspace,1.2 PS,SOAP, PS, SSL.

International Journal of Scientific & Engineering Research, Volume 7, Issue 12, December-2016ISSN 2229-5518Figure I: Average Response Time of LoadTesting ToolPOP3(S),IMAP(S),TCP.Figure I depict the graphical representation ofthe average response time of Jmeter,LoadComplete and WAPT.In Table I, basic information of selected loadtesting tool is given. This table describes theoperating system, browser support for the tools,license and pricing, initial release and currentrelease of these tools. It describes that all thesetools support multiple browsers and JMeter isplatform independent, whereas LoadCompleteand WAPT are only supported by windows.Table II: Average Response Time of LoadTesting Tools1Number ofVirtualUsers1210320JMeterLoadCompleteWAPT256 msec1683m sec922 msec1160 msec2100 msec853 msec50 msec110 msec110 msecAverage Response TIme5. Conclusion and Future ScopeLoad testing is concerned to analyzeperformance of web applications under differentnormal load testing conditions and anticipatedpeak conditions then analyze which factordegrade the performance. In this study ApacheJmeter, LoadComplete, and WAPT are analyzedand compared. The tests were performed at sameinstant of time and at same network speed. Fromthe above observation it is concluded that interm of response time WAPT has betterperformance than that of Jmeter andLoadComplete. Since, it is hard to determine theperformance on the basis of a single parameter.Thus, this work can be extended for more toolsand more parameters to provide more realisticand efficient results.IJSERReferencesTable II describes the average response time oftools for 1virtual user, 10 virtual users and 20virtual users and concluded that WAPT is betterthan that of Apache JMeter and LoadComplete.250020001500100050003301 VU10 VU20 VU[1] AngmoRigzin,SharmaMonika,"PerformanceEvaluation of Web BasedAutomation Testing Tools" IEEE 2014.[2] Burnstein Ilene, “Practical SoftwareTesting: A Process – Oriented Approach”Springer – Verlag New York.[3] Chandel Vandana, Patial Shilpa and GuleriaSonal, “Comparative study of Testing Tools:Apache Jmeter and Load Runner”International Journal of Computing andCorporate research (IJCCR) Volume 3,Issue 3, 3May 2013.[4] Huang Cheng-hui, Chan Huo Yan," A Toolto Support Automated Testing for WebApplication Scenario" IEEE InternationalConferenceonSystem,ManandCybernetics, October 2006.[5] Kaur Harpreet, Gupta Gagan, "ComparativeStudy for Automated Testing Tools:Selenium, Quick Test Professional andIJSER 2016http://www.ijser.org