Transcription
Using JMeterfor Testing aData CenterSiegfried GoeschlFreitag, 11. November 20111
“Siegfried, do you want to dosome performance testing?”“Well, what wouldyou like to test?”Freitag, 11. November 20112
“Some Data Centers with 300servers, MSSQL clusters, OracleRACs, large SAN storage, 1.5million registered users, 750call center agents, backup datacenter over dark fiber”Freitag, 11. November 20113
Freitag, 11. November 20114
The Challenge Ahead Performance testing on critical path‣ Prohibitive contractual penalties Lots of politics and finger-pointing‣ “Your performance tests are broken” Complex test scenarios‣ Often broken by new deploymentFreitag, 11. November 20115
Performance Test Scope End to end performance acceptance test Mandatory and client-witnessed test Strict acceptance criteria regarding‣ throughput‣ number of errors‣ severity of errorsFreitag, 11. November 20116
Subsystems Under Test Transaction web service interfaceprocessing 10,5 million transactions per day CRM web service interfaces being used by750 concurrent call center agents Public self-care web portal supporting 400concurrent & active web user sessionsFreitag, 11. November 20117
Performance Test TypesBaseline TestOne hour average loadStress TestOne hour peak loadEndurance Test24 hours maximum loadFreitag, 11. November 20118
Performance Test ToolsSubsystemPlannedDeliveredCRM Web ServicePublic Web PortalTransaction InterfaceFreitag, 11. November 20119
Testing Hardware Six dedicated load injectors‣ Windows 2008 Server‣ 3 physical and 3 virtual boxes‣ Dual quad-cores with 8 GB RAM No direct access‣ Citrix over RDP over RDPFreitag, 11. November 201110
What Is JMeter? Performance Test Tool‣ Web Testing (HTTP/HTTPS)‣ SOAP‣ JDBC, LDAP, SMTP, JMS, .Freitag, 11. November 201111
Why Using JMeter? It’s coming from the ASF Field-tested software No costs for virtual users Extensible using scripts and Java libraries Run from GUI, Ant task and command lineFreitag, 11. November 201112
JMeter Goodies Proxy server to record script Save request and response on error Constant Throughput Timer Summarizer for non-GUI tests Remote shutdown of running testsFreitag, 11. November 201113
Speed Up JMeter Use SOAP/XML-RPC Request Sampler‣ faster than WebService(SOAP) Sampler Avoid HTTPS when possible BeanShell scripts work best Minimize scripting Move large junks of scripting into librariesFreitag, 11. November 201114
JMeter Ant Integration jmeterjmeterhome " {jmeter.home}"testplan " {jmeter.test.script}"failureproperty "jmeter.test.failure"jmeterproperties " {jmeter.work.dir}/jmeter.properties"resultlog " {jmeter.test.result}" jvmarg value "-Xmx {jmeter.memory.max}" / jvmarg value "-Dsun.security.ssl.allowUnsafeRenegotiation true" / property name "jmeterengine.nongui.port" value " {nongui.port}"/ property name "log file" value " {jmeter.report.dir}/jmeter.log"/ property name "user.language" value "en"/ /jmeter Freitag, 11. November 201115
Things We Did To JMeter Three million WS requests over HTTPSrequests where the server closes thesocket after each request Sending up to 400 SOAP requests / sec Sending 400 KB SOAP requests‣ 70 MByte/sec over the wire Using up to 750 worker threads per JVM‣ 64-bit JVM with -Xmx 2048MFreitag, 11. November 201116
JMeter ReportingASP.NET WebsiteFreitag, 11. November 201117
JMeter Reporting No “Service Level Agreement” reporting HTML reports done using XSLT XSLT does not scale‣ We have huge JMeter result files‣ Unable to create formal test reportFreitag, 11. November 201118
JMeter ReportingResult of a 2,6 GByte JTL FileFreitag, 11. November 201119
JMeter SLA Report Merge multiple JTL files StAX parser to process large JTL file Statistic processing done by JAMon HTML export of JAMon model Available under ag, 11. November 201120
Freitag, 11. November 201121
JMeter SLA Report javaclassname ref "jmeter.class.path" sysproperty key "jmeter.thread.count" value "."/ sysproperty key "jmeter.thread.rampup" value "."/ sysproperty key "jmeter.thread.loops" value "."/ sysproperty key "jmeter.server.endpoint" value "."/ arg value " {jmeter.test.summary-report}"/ arg value " {jmeter.test.result}"/ /java Freitag, 11. November 201122
ASP.NET Web SitesFreitag, 11. November 201123
ASP.NET Web Sites Hidden fields and JavaScript parameters‣ ViewState‣ EventValidation‣ EventTarget ASP.NET security features Ajax callsFreitag, 11. November 201124
ASP.NET Web Sites Customer registration increased from 8 to32 HTTP requests over a few weeks “ViewState Errors” were popping up aftereach and every release Impossible to fix broken scripts underpressureFreitag, 11. November 201125
ASP.NET Web Sites Switched from JMeter to WAPT Pro withASP.NET module‣ 4-6 hours to write new script‣ Disabled security ASP.NET feature sincewe still got errors under heavy loadFreitag, 11. November 201126
Some More Thoughts Test your test tools No software installation Take two (or more) Automation is your friend Performance test for everyoneFreitag, 11. November 201127
Test Your Test Tools Tests tools are buggy‣ Multi-threading issues with soapUI Pro‣ Multi-threading issues with WAPT Pro‣ Replaced soapUI Pro with JMeterFreitag, 11. November 201128
No Software Installation Portable Apps on USB Stick‣ Run JMeter directly from USB Stick‣ Clone USB Stick to hard disk Have all your tools on the USB stick‣ Editor, Portable Git, .Freitag, 11. November 201129
Take Two (or more) A minimum of two share-nothing loadinjectors are required to prove that theservers is causing the performancebottleneck (and not your test tool) We used a maximum of five share-nothing load injectors simultaneouslyFreitag, 11. November 201130
Automation Is Your Friend Testing at night and over the weekend Test execution managed by Hudson Test failures triggers email notification Test protocols are copied to FTP serverFreitag, 11. November 201131
Performance Tests for Everyone All test scenarios are configured asHudson jobs‣Baseline, stress & endurance test Everyone can start and monitor aperformance test scenario over the webbrowser Even developers can run performancetestsFreitag, 11. November 201132
Performance Tests for EveryoneFreitag, 11. November 201133
Freitag, 11. November 201134
The few things youshould take homeFreitag, 11. November 201135
Lessons Learned Tests tools are buggy‣ Make sure that they work for you JMeter is perfectly able to run largeperformance tests reliably JMeter might not be handy for complexweb sites‣ Switched from JMeter to WAPT ProFreitag, 11. November 201136
Lessons Learned Performance tests are a valuable asset‣ Testing database failover‣ Testing data center replication & failover‣ Testing different server setups Creation and maintenance of complexperformance test suite is a project on itsownFreitag, 11. November 201137
Questions & AnswersFreitag, 11. November 201138
ResourcesJMeter SLA tJAMon APIhttp://jamonapi.sourceforge.net/JMeter ANT s/jmeter-ant-task.phpWAPT soapui.org/Freitag, 11. November 201139
The Software StackJMeterANTGitsoapUI ProHudsonGroovyWAPT ProPortable AppsJavaScriptFreitag, 11. November 201140
Switched from JMeter to WAPT Pro with ASP.NET module ‣ 4-6 hours to write new script ‣ Disabled security ASP.NET feature since we still got errors under heavy load Freitag, 11. November 2011 26
