WIXLIB

Question 7: AnswerWhat is the primary difference between a test plan, testdesign specification, and test procedure specification?project titleBefore executing a test Need to know what we are trying to testInputs / Expected outcomesHow to get ready and run the actual testsArtefacts for testing Each specified in its own document (IEEE 829)Test conditionsTest casesTest procedures

Question 7: AnswerWhat is the primary difference between a test plan, testdesign specification, and test procedure specification?project titleTest conditions Test Design SpecificationCondition: Something that can be testedApproach: “Throw the net wide”Identify as many conditions as possibleExhaustive testing is impossible Select a subset of all possible testsExamplesMeasuring branch coverage Test basis is the code itselfRequirements specification Table of contents can be the list of test conditions

Question 7: AnswerWhat is the primary difference between a test plan, testdesign specification, and test procedure specification?project titleTest cases Test Case SpecificationCase: Set of input values, preconditions, expected resultsDeveloped for a particular objective or test conditionCan cover a number of test conditionsE.g. Age, gender, enough credit?, place of birth, etc.ObjectiveTo assess that the system does what it is supposed to doGiven the conditions, does it behave correctly?

Question 7: AnswerWhat is the primary difference between a test plan, testdesign specification, and test procedure specification?project titleTest procedures Test Procedure SpecificationProcedure: Sequence of actions for the execution of a testTest scriptManual test scriptFormed into a test execution scheduleDescription of the order of the test proceduresBy whom do the tests need to be run?Useful for prioritising tests Ensure best testing is done in the time available

Question 7: AnswerWhat is the primary difference between a test plan, testdesign specification, and test procedure specification?project titleExample: Buying movie ticketsConditions The things we can test (age, time, discount)Cases Combination of conditions to assess system behaviour (rules)Procedure Sequence of actions to execute test (step-by-step)

Question 7: AnswerWhat is the primary difference between a test plan, testdesign specification, and test procedure specification?project title

Question 7: AnswerWhat is the primary difference between a test plan, test designspecification, and test procedure specification?project titlea.b.c.d.The test plan describes one or more levels of testing, the test designspecification identifies the associated high-level test cases and a testprocedure specification describes the actions for executing a testThe test plan is for managers, the test design specification is for programmers andthe test procedure specification is for the testers who are automating the testsThe test plan is the least thorough, the test procedure specification is the mostthrough and the test design specification is midway between the twoThe test plan is finished in the first third of the project, the test design specificationis finished in the middle third of the project and the test procedure specification isfinished in the last third of the project

project titleEntry and Exit Criteria

Question 8Entry criteria for testing means that the companymanagement gave their OK to the developmentteamtoproject titlestart the test activitiesa. Trueb. False

Question 8: AnswerEntry criteria means that the company management gavetheir OK to the dev. team to start the test activitiesproject titleEntry criteriaDefines when to start testingCentral question: Do we have enough to conduct a meaningful test effort?Typical considerations

Question 8: AnswerEntry criteria for testing means that the companymanagement gave their OK to the developmentteamtoproject titlestart the test activitiesa. Trueb. False

Question 9The ISTQB Foundation Syllabus established a fundamental testprocess where test planning occurs early in the project, whileproject titletest execution occurs later. Which of the following elements ofthe test plan, while specified during test planning, areassessed during test execution?a.b.c.d.Test tasksEnvironmental needsExit criteriaTest team training

Question 9: AnswerWhich of the following elements of a test plan, whilespecified during test planning, are assessed duringtestproject titleexecution?Purpose of testingVerify some aspect of the system / Reveal faults in the implementationWhat do we define prior to test execution?Test tasks: What is to be done and when?Environmental needs: What requirements do we have for the test environment?Entry and exit criteria: When to start and when to end the test effort?

Question 9: AnswerWhich of the following elements of a test plan, whilespecified during test planning, are assessed duringtestproject titleexecution?What do we assess during to test execution?The results of the test against the pre-defined objectivesWe are interested in answering the following questionsHow did the test go?Did it go according to plan? Why, why not?Do we need to run more tests?

Question 9: AnswerWhich of the following elements of a test plan, whilespecified during test planning, are assessed duringtestproject titleexecution?Exit criteria When to stop testing? (Definition of enough)Typical considerations

Question 9: AnswerThe ISTQB Foundation Syllabus established a fundamental testprocess where test planning occurs early in the project, whileproject titletest execution occurs later. Which of the following elements ofthe test plan, while specified during test planning, areassessed during test execution?a.b.c.d.Test tasksEnvironmental needsExit criteriaTest team training

Question 9: ExampleExample using Entry and Exit criteriaWant to test login functionality for an imaginary websiteproject titleWe write test cases for two different scenariosUser already registered / User not registeredID12Test CaseTest if registered user is ableto log in successfullyTest if unregistered user is notable to log inPreconditions Input Test DataCorrectUser must be NoneIncorrectpasswordProcedure1. Enter input usernameand password2. Click "Login"1. Enter input usernameand password2. Click "Login"ExpectedResultsLoginsuccessfulLogin failed

Question 9: ExampleExample using Entry and Exit criteriaUse entry and exit criteria to assess the test effortEntry criteriaTesting environment established?YesGraphical user interface in placeWe choose manual testingAdequate test data is available?Valid username / Valid passwordInvalid username / Invalid passwordproject title

Question 9: ExampleExample using Entry and Exit criteriaAssume we run the two tests, and get the following:Must assess tests based on exit criteriaproject title

Question 9: ExampleExample using Entry and Exit criteriaUse entry and exit criteria to assess the test effortExit criteriaAll test cases (100 %) have been executed?Yes Both test 1 and test 2 have been executedFailed cases have a satisfactory resolution?Yes Developers will fix the discovered defectDefects were documented and reported?Yes Defect revealed by test 2 has been documentedNew tests will be run once developers fix the discovered defectproject title

Question 10Consider the following exit criteria which might be found in a testplan. Which of these belong in an acceptance test plan?1.2.3.4.5.a.b.c.d.project titleNo known customer-critical defectsAll interfaces between components tested100 % code coverage of all itemsAll specified requirements satisfiedSystem functionality matches legacy system for all business rulesAll statements belong in an acceptance test planOnly statement 1 belongs in an acceptance test planOnly statements 1, 2 and 5 belong in an acceptance test planOnly statements 1, 4 and 5 belong in an acceptance test plan

Question 10: AnswerWhich of the given exit criteria belong in an acceptancetest plan?project titleAcceptance testingValidation testing with respect to the usersRequirements, business processesTypically assesses aspects such asThe system’s adherence to the requirements specificationDefects that may be critical to the customerFunctionality in accordance with the business rules?Whether or not the system is fit for use

Question 10: AnswerConsider the following exit criteria which might be found in a testplan. Which of these belong in an acceptance test plan?1.2.3.4.5.a.b.c.d.project titleNo known customer-critical defectsAll interfaces between components tested100 % code coverage of all itemsAll specified requirements satisfiedSystem functionality matches legacy system for all business rulesAll statements belong in an acceptance test planOnly statement 1 belongs in an acceptance test planOnly statements 1, 2 and 5 belong in an acceptance test planOnly statements 1, 4 and 5 belong in an acceptance test plan

Question 11During test execution, the test manager describes thefollowing situation to the project team:project title 90 % of the test cases have been run.20 % of the test cases have identified defects.110 defects have been found.100 defects have been fixed and have passed confirmationtesting. Of the remaining 10 defects, project management has decidedthat they do not need to be fixed prior to release.

Question 11Which of the following is the most reasonableinterpretation of this test status report?project titlea. The remaining 10 defects should be confirmation tested prior toreleaseb. The remaining 10 % of test cases should be run prior to releasec. The system is ready for release with no further testing ordevelopment effortd. The programmers should focus their attention on fixing theremaining known defects prior to release

Question 11: AnswerWhich of the following is the most reasonableinterpretation of this test status report?project titleTest progress monitoringGathering detailed test dataTest status reportingAnalysing available information and metrics to support conclusionsHave exit criteria been met?Effectively communicating the findings to various stakeholdersEnsure stakeholders understand the results of a test period

Question 11: AnswerWhich of the following is the most reasonableinterpretation of this test status report?project titleWhat we know from the given status report90 % of the test cases have been run20 % of the test cases have identified defects110 defects have been found100 defects have been fixed, and have passed confirmation testing10 defects remaining do not need to be fixed prior to releaseWhat do these points indicate?

Question 11: AnswerWhich of the following is the most reasonableinterpretation of this test status report?project title90 % of the test cases have been run10 % of the test cases have not been run yetFollow-up question: Given that 10 % of cases remain untested, are we finished?20 % of the test cases have identified defects110 defects have been foundFrom the 90 % of the run test cases, 20 % of those found defectsThe total number of defects in this portion were 110

Question 11: AnswerWhich of the following is the most reasonableinterpretation of this test status report?project title100 defects have been fixed, and have passed confirmation testingIdentified 100 defects as customer-criticalWe needed to fix these prior to releaseThe 10 defects remaining do not need to be fixed prior to releaseWe fixed 100 out of 110 defectsThe remaining 10 defects are not regarded customer-criticalWe leave these defects untouched

Question 11: AnswerWhich of the following is the most reasonableinterpretation of this test status report?project titlea. The remaining 10 defects should be confirmation tested prior toreleaseb. The remaining 10 % of test cases should be run prior toreleasec. The system is ready for release with no further testing ordevelopment effortd. The programmers should focus their attention on fixing theremaining known defects prior to release

Question 12The purpose of criteria is to define when tostop testing, such as at the end of a test level orwhenaproject titleset of tests has a specific goal.

Question 12: AnswerThe purpose of criteria is to define when tostop testing, such as at the end of a test level orwhenaproject titleset of tests has a specific goal.EXIT (criteria)

Question 13The metrics for test progress monitoring can be collectedboth manually and automaticallyproject titlea. Trueb. False

Question 13: AnswerThe metrics for test progress monitoring can be collectedboth manually and automaticallyproject titleTest progress monitoringProvide feedback on how the test effort is goingVisible information about the test resultsMeasure the status of testingCan use metrics to assess progress Evaluated against exit criteriaTest case execution Number of test cases run / not run, passed / failedTest coverage Requirements / Risks / CodePercentage of work done Case and environment preparation

Question 13: AnswerThe metrics for test progress monitoring can be collectedboth manually and automaticallyproject titleHow to collect metrics for test progress monitoring?ManuallyCollect information about the test progress by handE.g. Using spreadsheets, checklists, etc.AutomaticallyUsing progress monitoring softwareMetrics generated directly from the test suitesDifferent test tools provide different features

Question 13: AnswerThe metrics for test progress monitoring can be collectedboth manually and automaticallyproject titleDifferent ways of representing metrics and test progress

Question 13: AnswerThe metrics for test progress monitoring can be collectedboth manually and automaticallyproject titlea. Trueb. False

Question 14Pair the following roles with their typical activitiesproject titleTesterTest LeaderEvaluates the results of the execution of tests: Pass or failEvaluates the exit criteria and gives recommendations based onit: Continue testing or stopIntroduces metrics for measuring the test progressTest data: Acquires it and prepares itWrites test summary reports for managementWrites automated tests

Question 14: AnswerPair the following roles with their typical activitiesproject titleTesterTestLeaderEvaluates the results of the execution of tests: Pass or failEvaluates the exit criteria and gives recommendations basedon it: Continue testing or stopIntroduces metrics for measuring the test progressTest data: Acquires it and prepares itWrites test summary reports for managementWrites automated tests

project titlePart II: Exercises andOpen-ended questions

Exercise 1Describe briefly what is meant by the following testapproaches (strategies)project titlea.b.c.d.e.f.g.Analytical approachModel-based approachMethodical approachProcess- or standard-compliant approachDynamic and heuristic approachConsultative approachRegression-averse approachIs one approach better than the other? Why, why not? Whichdo you prefer?

Exercise 1(a): AnswerAnalytical approachUse of formal / informal analytical techniqueproject titleFactors strongly affecting the testing environmentRisk-based strategyPerform risk analysis using project documents and stakeholder inputPlanning, estimation, designing, and prioritising tests based on riskRequirements-based strategyAnalysis of the requirements specificationBasis for planning, estimation, design

Exercise 1(b): AnswerModel-based approachTests designed based on models of the object functionalityproject titleCritical system behaviourEmphasis on identification and selection of the appropriate modelPreventive test approachExamplesCan build mathematical models for loading/response for e-commerce serversDesign tests based on the modelsCheck if behaviour of the system conforms to the predicted behaviour (from model)

Exercise 1(c): AnswerMethodical approachAdhere to a pre-planned, systematised approachDeveloped in-house, assembled from various conceptsFollowing a specific methodTests are designed, executed and implemented in accordanceExamplesAdherence to certain checklistsFailure-based (error checking, fault-attacks)Experience-basedproject title

Exercise 1(d): AnswerProcess- or standard-compliant approach“Go by the book” as opposed to “do it your own way”Uses externally developed industry standardsDesign and implement test assets based on theseLittle (or no) customisationExamplesAdhering to IEEE 829 standardsAdhering to agile methodologiesE.g. Extreme programming (XP)project title

Exercise 1(e): AnswerDynamic and heuristic approachHeuristic (techniques)Approach to problem solving that employs a practical methodsNot guaranteed to be optimal, but sufficient for intermediate goalsTypically applied during the later stages of testingProblem: Exhaustive testing is impossibleSolution: Finding as many defects as possibleExamplesExploratory testingproject title

Exercise 1(f): AnswerConsultative approachRely on a group of non-testers to guide / perform test effortproject titleSeek advice and guidance from externalsBusiness domain expertsTechnical experts (e.g. security experts)Outside the test teamExamplesAsking users for what to testAsking developers to develop the tests

Exercise 1(g): AnswerRegression-averse approachTechniques to manage the risk of regressionproject titleGoal: Avoid (re-)occurrence of defectsCreate effective regression tests to run when anything changesRe-run every test to ensure nothing has been brokenPracticesAutomating functional tests prior to release / Re-use existing test materialExampleHaving a standard set of test data for which a screen needs to pass

Exercise 1: AnswerIs one approach better than the other? Why, why not?Some strategies are preventive (prior)project titleAnalytical test strategies involve upfront analysis of the problem areaTend to identify test basis prior to test executionSome strategies are reactive (during)Dynamic test strategies focus on the test execution periodEnable the identification of defect (clusters) that may be hard to anti

the test procedure specification is for the testers who are automating the tests c. The test plan is the least thorough, the test procedure specification is the most through and the test design specification is midway between the two d. The test plan is finished in the first third of the project, the test design specification