Transcription
Model Based Testingfor VoIP PhonesDinesh Patil, AvayaJani Koivulainen, ConformiqJagadish Vellanki, IdeabytesSeptember 2010
What is Model Based Testing?Synthesizestest stimuliModelTest in actualframeworkRealSystem The basic idea in Model Based Testing (MBT) is tocheck the conformance of a system implementation (SUT)to the specification by modeling its expected behavior Directly from this model, the user selects the algorithmsto use in the Automated Test Design (ATD) tool ATD automatically designs and generates black boxfeature test scripts and proper test outcomes (test oracle)Avaya Inc. – Proprietary. Use pursuant to the terms of your signed agreement or Avaya policy.2
Why Avaya Decided to UseMBT We realized that improvingour testing by just bringingin new people was no longeran option, so we had tochange our test designmethodologyTestCoverage# of Features MBT (Model Based Testing) and ATD (AutomatedTest Design) seemed like a good way to significantlyboost our testing capability Conformiq Designer software was selected Within six months of implementation we establishedMBT /ATD as an essential part of our testing processAvaya Inc. – Proprietary. Use pursuant to the terms of your signed agreement or Avaya policy.3
Productivity Gains Using MBT No manual selection and enumeration of test cases No need to produce test data No debugging of incorrect test cases Improved test cases and coverage Reports missed requirement tests Faster to update model than test scripts Maps test cases to requirements Consistent test case design Automatically matches tests to requirements No test automation backlog
Avaya Automated TestingEnvironmentModelTester HostWindows ServerSystemUnder TestATE (Groovy)GeneratedTest riesConformiqDesigner TextualTest PlansExecutionLogsUISNMPNetworkConfigure(Phone Network)Avaya Inc. – Proprietary. Use pursuant to the terms of your signed agreement or Avaya policy.5
Avaya VoIP Test SystemArchitectureOur Groovy-based Automated Test Environment (ATE) is verycomplex and includes an extensive set of functionalities forinteracting programmatically with our VoIP phonesAvaya Inc. – Proprietary. Use pursuant to the terms of your signed agreement or Avaya policy.6
MBT Tool Operation Manually create systemmodel Automatically designstest input and expectedoutput with data andtimer handling Generates test reports Renders test scripts inGroovy
Automated Test DesignGenerationAvaya Inc. – Proprietary. Use pursuant to the terms of your signed agreement or Avaya policy.8
Generated Test Scripts In order to generatedirectly executable testcases for ATE,Conformiq Designer uses a Groovyscripting backend This Groovy-fragment shows what a typical test script looks like:// Define phone object i.e. the "end point"endpoint1 define(type:'spice',alias:'endpoint1',ip:endpoint1 endpoint:endpoint1,extension:endpoint1 extension,password:'1234',gatekeeper:mygk1)// Navigate to contacts application using the phone objectbutton(endpoint1,"AddressBook")// press "New" soft key on phonebutton(endpoint1,"SoftKey0")Avaya Inc. – Proprietary. Use pursuant to the terms of your signed agreement or Avaya policy.9
Testing Efficiency ResultsTestAreaNo. ofManualTest CasesNo. ofConformiqTest CasesNo. ofManualTest StepsNo. ofConformiqTest l Effortin HoursConformiqEffort in honeApplications8274281.93Avaya Inc. – Proprietary. Use pursuant to the terms of your signed agreement or Avaya policy.10
Automated Test DesignBenefitsHigher Productivity in Test DesignImproved Test CoverageSimplified Test MaintenanceModel ReuseSimplified Test HarnessSpecification Issues Are Found EarlierTest Automation Backlog Is EliminatedAvaya Inc. – Proprietary. Use pursuant to the terms of your signed11agreement or Avaya policy.
Thank YouQuestionsAvaya Inc. – Proprietary. Use pursuant to the terms of your signed agreement or Avaya policy.12
Test Area No. of Manual Test Cases No. of Conformiq Test Cases No. of Manual Test Steps No. of Conformiq Test Steps Test Case Coverage Gain Network Config 400 1440 5445 93000 3.60 Phone Avaya Inc. - Proprietary. Use pursuant to the terms of your signed agreement or Avaya policy. 10 Apps 544 1360 6859 85000 2.50 Test Area Manual Effort in .
