Transcription
Budapest, 26-28 October 2016NEXT GENERATION TEST AUTOMATIONPresented by Stephan Schulz, CTO of Conformiq All rights reserved
About Conformiq Founded in 1998 in Finland Software & support services Conformiq 360 Test Automation 3rd generation of software General purpose test designand automation tools Worldwide locations San Jose, CA, USA (HQ) Helsinki, Finland (R&D) Bangalore, India (R&D) Paris, Hyderabad, Toronto (sales & support) Partner with SDLC tool providers andtesting services companies Customers worldwide Fortune 100 Enterprise & Networking markets BFSI, Telecom service providers2 All rights reserved2015TOP20CIO Review20 Most PromisingSoftware TestingSolution Providers 2015
Food for Thought What percentage of tests do you execute today in anautomated way? Regression versus progression? What is your current requirement coverage? How confident are you that all of your regressiontests actually need to be executed (at all)?3
Data from a Previous Poll (Enterprise IT ‐ EU) What percentage oftests do you executetoday in an automatedway? What is your currentrequirement coverage?4More than 80%10%Between 41% and 80%12%Between 20% and 40%15%Less than 20%56%We do not automate7%More than 81%28%Between 61% and 80%35%Less than 60%28%Not sure10%
Some Related Quotes “We have 3 testers per developer and defects still slipthrough.” Cardinal Health Test Lead “I have 30,000 test cases and my testers can not tellme what they cover.” Huawei Test Director “I was surprised about the throughput of one of mytesters – until I decided to review his test scripts ”Enterprise IT Test Manager5
In a Perfect World: What would I want? 100%Increase productivityIncrease and optimize test coverageFull and complete integration6
It Takes More than Automating Test erationChallenges Requirements DocumentationDeployment costRequired skill setMaintenanceTest qualityCycle timeContinuous integrationScriptsProductchanges7
Evolution of Software TestingImproved TestingEfficiencyMBTTest ModelsTest FrameworksKeyword DrivenTest ScriptsCapture/ReplayManual Testing8
Combine MBT with Test Execution AutomationImproved TestingEfficiencyMBT Automate testdesignTest ModelsTest FrameworksKeyword DrivenTest ScriptsCapture/ReplayManual Testing9 Shorten path to testexecution automation Simplify MBT adoption
What is Model Based Testing?TargetsTestsModel ofFunctionality 10AutomaticgenerationRealFunctionalityCapture operation of application in a model instead of writing testsGenerate tests and test data based on your targetsHigher coverage & faster test execution from automatic test designScripts, documentation, and coverage automatically in sync
Next Generation Test AutomationALM /Requirement ManagementALM /Test ManagementMBTUpdate documentationautomaticallyTrack requirement changesautomaticallyTest ExecutionExisting AssetsCreate modelsautomatically11Maintain a model andupdate everything elseautomaticallyExecute testsautomatically
Model Based Testing is DisruptiveFunctionality vs PathApplication vs Environment95%Coverage vs Number of Tests42Abstraction vs DetailAutomated vs Manual DesignTest 1: Test 2: Test 3: Flexible vs Fixed Test SetsTest 1: Test 2: Test 3:
Complete Automation of End2End Testing ProcessManual ManagementModelingTest omationAssets
Model Capture from Existing Assets sub-diagramactivitytest data eventdecision Model across all interfacesavailable for testingImport interface informationfrom UI recordings, API specs(WSDL/XSD, WADL/JSON, ) 1Model requirements inworkflows to be testedImport Gherkin feature files ormanual tests from spreadsheetsImport models from flowchartor other BPMN tools
Review Automatically Generated TestsTestTargetsTest CoverageTestSteps & DataGenerated Tests Validate assumptions with all stakeholders before test execution Know what you cover and what you don’t!152
Integrate with all [Existing] SDLC ToolsALM /Requirement ManagementALM /Test ManagementMBTExisting Tests,Recordings & User StoriesAutomatic Test ExecutionVersion ControlGherkin163
It is not a Dream or Vision ‐ it is Very Real! “From 3 Stars Rating to 5 Stars in the App Store”Javier Moscardó ‐ Spotify ‐ UCAAT 2015 (best presentation) “Requirements and Challenges with Advanced TestAutomation, The Industry Perspective”Sigrid Eldh ‐ Ericsson ‐ UCAAT Keynote 2015 “The World's Largest MBT Deployment Transforms Testing forQuality at Agile Speed”Clark Cochran ‐ Conformiq ‐ STC 2015 “Shifting Test Automation to the Left of SDLC”Ish Kumar – Tech Mahindra – Blog 201517
What will get us to the “6 star rating”? Next step in evolution of automated testing isappearing on the horizon already What is likely to fail? What is failing and why? Is it a symptom or defect? And more 18
Q&A and DiscussionThank you!stephan.schulz@conformiq.com19
BACKUP All rights reserved
About Conformiq 2 Founded in 1998 in Finland Software & support services Conformiq 360 Test Automation 3rd generation of software General purpose test design and automation tools Worldwide locations San Jose, CA, USA (HQ) Helsinki, Finland (R&D) Bangalore, India (R&D) Paris, Hyderabad, Toronto (sales & support)
