Rational TestManager User’s Guide

Transcription

Rational Software CorporationRational TestManager USER’S GUIDEVERSION: 2002.05.00PART NUMBER : nal.com

IMPORTANT NOTICECOPYRIGHTCopyright 2000-2001, Rational Software Corporation. All rights reserved.Part Number: 800-025139-000Version Number: 2002.05.00PERMITTED USAGETHIS DOCUMENT CONTAINS PROPRIETARY INFORMATION WHICH IS THE PROPERTY OFRATIONAL SOFTWARE CORPORATION (“RATIONAL”) AND IS FURNISHED FOR THE SOLEPURPOSE OF THE OPERATION AND THE MAINTENANCE OF PRODUCTS OF RATIONAL. NO PARTOF THIS PUBLICATION IS TO BE USED FOR ANY OTHER PURPOSE, AND IS NOT TO BEREPRODUCED, COPIED, ADAPTED, DISCLOSED, DISTRIBUTED, TRANSMITTED, STORED IN ARETRIEVAL SYSTEM OR TRANSLATED INTO ANY HUMAN OR COMPUTER LANGUAGE, IN ANYFORM, BY ANY MEANS, IN WHOLE OR IN PART, WITHOUT THE PRIOR EXPRESS WRITTENCONSENT OF RATIONAL.TRADEMARKSRational, Rational Software Corporation, the Rational logo, Rational the e-development company,ClearCase, ClearQuest, Object Testing, Object-Oriented Recording, Objectory, PerformanceStudio,PureCoverage, PureDDTS, PureLink, Purify, Purify'd, Quantify, Rational Apex, Rational CRC, RationalPerformanceArchitect, Rational Rose, Rational Suite, Rational Summit, Rational Unified Process, RationalVisual Test, Requisite, RequisitePro, SiteCheck, SoDA, TestFactory, TestMate, TestStudio, and The RationalWatch are trademarks or registered trademarks of Rational Software Corporation in the United States andin other countries. All other names are used for identification purposes only, and are trademarks orregistered trademarks of their respective companies.Microsoft, the Microsoft logo, the Microsoft Internet Explorer logo, DeveloperStudio, Visual C , VisualBasic, Windows, the Windows CE logo, the Windows logo, Windows NT, the Windows Start logo, andXENIX are trademarks or registered trademarks of Microsoft Corporation in the United States andother countries.Java and all Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in theUnited States and other countries.FLEXlm and GLOBEtrotter are trademarks or registered trademarks of GLOBEtrotter Software, Inc.Licensee shall not incorporate any GLOBEtrotter software (FLEXlm libraries and utilities) into any productor application the primary purpose of which is software license management.PATENTU.S. Patent Nos.5,193,180 and 5,335,344 and 5,535,329 and 5,835,701. Additional patents pending.Purify is licensed under Sun Microsystems, Inc., U.S. Patent No. 5,404,499.GOVERNMENT RIGHTS LEGENDUse, duplication, or disclosure by the U.S. Government is subject to restrictions set forth in the applicableRational Software Corporation license agreement and as provided in DFARS 277.7202-1(a) and277.7202-3(a) (1995), DFARS 252.227-7013(c)(1)(ii) (Oct. 1988), FAR 12.212(a) (1995), FAR 52.227-19, or FAR227-14, as applicable.WARRANTY DISCLAIMERThis document and its associated software may be used as stated in the underlying license agreement.Rational Software Corporation expressly disclaims all other warranties, express or implied, with respect tothe media and software product and its documentation, including without limitation, the warranties ofmerchantability or fitness for a particular purpose or arising from a course of dealing, usage, or tradepractice.

ContentsPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvAudience. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xvOther Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xvContacting Rational Technical Publications . . . . . . . . . . . . . . . . . . . . . . . . . .xvContacting Rational Technical Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviPart 1: Using TestManager to Manage Testing Projects1Introducing Rational TestManager. . . . . . . . . . . . . . . . . . . . . . . . . . . 1What Is Rational TestManager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1TestManager Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2Planning Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Designing Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Implementing Tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Executing Tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Evaluating Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36678TestManager and Other Rational Products . . . . . . . . . . . . . . . . . . . . . . . . . . 8The Rational Unified Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Projects and the Rational Administrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Automated Test Scripts and Rational Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Automated Test Scripts and Rational TestFactory . . . . . . . . . . . . . . . . . . . . . . . 10Component Testing and Rational QualityArchitect . . . . . . . . . . . . . . . . . . . . . . . 11Requirements and Rational RequisitePro . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Model Elements and Rational Rose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Defects and Rational ClearQuest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Reports and Rational SoDA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12TestManager and Extensibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13Defining Custom Test Script Types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13Defining Custom Test Input Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13Defining Custom Test Script Types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14Functional and Performance Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14Contentsiii

Functional Testing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14Performance Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Virtual Testers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15Virtual Testers in Functional Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Virtual Testers in Performance Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16Local and Agent Computers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16The TestManager Environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Suites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18Starting Rational TestManager. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19Logging On toTestManager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19Starting Other Rational Products and Components from TestManager . . . . . . . 20The TestManager Main Window. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21Test Asset Workspace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22Other TestManager Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262Planning Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27About Test Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27Identifying What to Test by Using Test Inputs . . . . . . . . . . . . . . . . . . . . . . . .28Built-in Test Input Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29Custom Test Input Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31Creating a Test Plan. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32Creating Test Plans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33Properties of a Test Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34Organizing Test Cases with Folders. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36Inheriting Iterations and Configurations from a Test Case Folder. . . . . . . . . . . . 37Creating Test Cases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38Properties of a Test Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Specifying the Owner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Defining the Configurations to Test. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Inheriting Test Case Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Specifying When to Run Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Test Impact Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .394040495052Managing Test Assets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .58Upgrading from Previous Versions of Rational TestManager . . . . . . . . . . . . . . .Exporting and Importing Test Assets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Copy, Cut, and Paste Test Assets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Software Configuration Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ivContents58596363

3Designing Tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67About Designing Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .67Benefits of Good Test Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68Specifying the Testing Steps and Verification Points . . . . . . . . . . . . . . . . . . .70Specifying Conditions and Acceptance Criteria of Test Cases . . . . . . . . . . .72Example of a Test Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .734Implementing Tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75About Implementing Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .75Test Scripts Window. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76Implementing Test Cases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .77Built-in Test Script Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77Custom Test Script Types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78Suites Created in TestManager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80Calling Test Script Services from Test Scripts . . . . . . . . . . . . . . . . . . . . . . . .81Test Script Services and Test Script Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81Test Script Services and TestManager. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82Creating Manual Test Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .83Importing a Manual Test Script from Another Project or a File . . . . . . . . . . . . . 84Creating a Manual Test Script from a Test Case Design . . . . . . . . . . . . . . . . . . 84Creating a Manual Test Script in Rational ManualTest. . . . . . . . . . . . . . . . . . . . 84Associating an Implementation with a Test Case. . . . . . . . . . . . . . . . . . . . . .88Implementing Tests as Suites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .90Activities Common to Performance and Functional Testing . . . . . . . . . . . . . . . . 92Changing the Settings of an Agent Computer . . . . . . . . . . . . . . . . . . . . . . . . . . 95Creating a Suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95Opening a Suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96Editing a Test Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97Editing the Properties of a Suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98Replacing Items in a Suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99Editing the Run Properties of Items in a Suite . . . . . . . . . . . . . . . . . . . . . . . . . 100Editing Information for All User and Computer Groups . . . . . . . . . . . . . . . . . . 100Editing Settings for Virtual Testers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101Initializing Shared Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108Printing and Exporting a Suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110Saving a Suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110Contentsv

5Executing Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111About Running Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .111Built-In Support for Running T

Rational TestManager and Performance Testing . . . . . . . . . . . . . . . . . . . . .244 Creating Test Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244 Planning Performance Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .245