WIXLIB

SIXTEENTH ANNUALPACIFIC NORTHWESTSOFTWARE QUALITYCONFERENCEJoint withASQ Software Division’sEIGHTH INTERNATIONALCONFERENCE ON SOFTWAREQUALITYOCTOBER 13 - 14, 1998Oregon Convention CenterPortland, OregonPermission to copy without fee all or part of this material, except copyrighted material asnoted, is granted provided that the copies are not make or distributed for commercial use.

TABLE OF CONTENTSPreface. ivConference Officers/Committee Chairs . vConference Planning Committee. viKeynote Address—October 13Characteristics of the Learning Organization . 1Tom DeMarco, Atlantic Systems GuildKeynote Address—October 14Process Out, Quality In. 12Tom Gilb, Results Planning LimitedManagement Track—October 13Critical Chain—Doing Development Faster with Quality . 26Richard E. Zultner, Zultner & CompanyAvoiding Litigation: Reflections of an Expert Witness . 48Brian Lawrence, Coyote Valley SoftwareA Cost Estimation Based ApproachTo Quantify Software Risk Evaluation Results . 57Peter Hantos, XeroxFrom VICTIM to VICTOR: The Trials & Tribulations of aNeophyte Software Testing Team . 66Elizabeth A. Adams, Louisiana Workers’ Compensation CorporationTransferring Software Development Best Known Methods (BKMs)Between Generational Product Lines . 75James R. Bindas, Intel CorporationAnticipating & Mitigating the Professional Challengeto Independent Verification & Validation. 84James B. Dabney, Intermetrics, Inc.James D. Arthur, Virginia Tech.Metrics Track—October 13Selecting & Designing Metrics by Example: Two Successful Paradigms . 93Claire Lohr, Lohr SystemsExperiences Implementing Software Measurement. 97Beth A. Layman, & Sharon L. Rohde, Lockheed Martin Mission Systems

Testing, Requirements, and Metrics . 107L. Rosenberg, L. Huffman, Unisys; T. Hammer, NASAManagement & Technical Opportunities & Barriers toApplying Statistical Continuous Improvement to Software Quality. 123Mervin E. Muller, Ohio State UniversityMetrics and the Financial Health of Software Organizations . 142Paul Doherty, Oregon Graduate InstituteDon Springer, University of PortlandSuccessful Strategies for Implementing Software Metrics at Intel . 156Paul Dittman, Jeanne Yuen-Hum, Intel Corp.TestingTrack—October 13Defining Test Policies & Standards . 163Cheryl Moore, FedEx CorporationA Testing Strategy for Outsourced Multi-Platform Software Solutions . 177Kersti Nogeste, SMS Consulting Group Pty Ltd, AustraliaTowards Quality Programming in theAutomated Testing of Client/Server Applications. 189Huey-Der Chu, National Defense Management College, TaiwanJohn Dobson, Univ. of Newcastle upon Tyne, U.K.The Automated Build Verify Test: A Valuable Time Saver. 216Bruce Kovalsky, Fidelity Technology SolutionsAn Excel-Based Test Harness for Windows Software . 224Robert Bales, Tektronix, Inc.Assessment of COTS Products from an Operating Systems Perspective. 235Ronald J. Leach, Howard UniversityTrack 4—October 13Making Test Cases from Use Cases Automatically. 244Robert M. Poston, AonixProcess - Experience—October 14Strategic Planning Process—How to Plan Improvement. 266Mary Sakry, Neil Potter, The Process GroupA Phased Approach to the PSP. 289Jeff S. Holmes, Motorola Cellular Infrastructure GroupBonnie E. Melhart, Texas Christian UniversitySuccessful Implementation of ISO 9001/TickIT in a Software Development Company . 300Cecilia M. Yourshaw, VTLS Inc.A Problem-Based Approach to Software Process Improvement: A Case Study. 310Johanna Rothman, Rothman Consulting GroupI’ve Been Asked to Review This Specification: Now What Do I Do? . 317Karen Bishop-Stone, Testware Associates, Inc.IEEE/EIA 12207 as the Foundation for Enterprise Software Processes . 326James W. Moore, The MITRE Corporation

Process - CMM—October 14Stepwise Improvement of your Test Processes Using TPI .334Martin Pol, IQUIP Informatica B.V., The NetherlandsUsing the Software CMM With Judgment:Small Projects & Small Organizations . 350Mark Paulk, Software Engineering InstituteProcess Synergy: Using ISO 9000 and the CMM in aSmall Development Environment. 362Sharon E. Miller, Northstar Consulting GroupJUSE Papers:Quality Assurance Activities in the Software Development Center, Hitachi Ltd. . 372Hitoshi Kiharra, Senior Engineer, Quality Assurance Department, Software Development Center, Hitachi,Ltd.Introduction and Evaluation of SPA Method for Telecommunication Software Development. 385Ichiro Kamata, Sadayuki Higasi, Fujio Moriyama, NTT Communicationware Corporation, JapanA Process Improvement Method Using Software Quality Analysis Tools . 398Taeshi Tanaka, Minoru Aizawa, Hideto Ogasawara, Takumi Kusanagi, Atsushi Yamada, HideoNakamura,Systems & Software Research Laboratories, Toshiba CorporationExample of Field Quality Improvement by Raising the Level of CMM. 412Mitsuru Ishio, Quality Assurance Department, NEC Communication SystemsProcess - Models—October 14Developing a Software Quality Modelto Improve Supplier Performance . 422Craig Smith, Motorola Cellular Infrastructure GroupImproved Software Quality by Adopting Control Charts . 430Anders Subotic & Niclas Ohlsson, Linkoping University, SwedenAn Integrated Software Audit Process Model toDrive Continuous Improvement . 445Neda L. Gutowski, Motorola Cellular Infrastructure GroupThe Effects of the Year 2000 Problemon the Wintel Duopoly’s Control of Internet Commercein the International Marketplace for Multimedia Mindshare – or –How Do We Get These Damn Things to Work? . 457Lincoln Spector, Computer Journalist, Columnist, HumoristTesting—October 14Hierarchical Organization of Test Cases. 460Michael Ensminger, Partes CorporationSo You Think You Know Objects?. 470Ray Lischner, Tempest SoftwareImproving the Test Process using New Technology. 476Fareed K. Shaikh, Lawrence E. Niech, Automatic Data Processing

Experience Report: Comparing an Automated Conformance Test Development Approachwith a Traditional Development Approach . 485Alan Goldfine, Gary Fisher,Lynne Rosenthal, NISTNuts’n Bolts Experiences in Code Coverage Analysis . 491Pemmaraju S. Rao, Richard Vireday, Intel Corp.Beyond Coverage Analysis —Time Optimization of Regression Testing . 507Roy Trammell, Performance Tools GroupImproving the Joint Dewvelopment Relationship. 523Karen S. KingThird-Party Registrars’ Audits - for Better or for Worse?. 535Robert C. Bamford, William J. Deibler II, Software Systems Quality ConsultingHybrid Multi-Model Assessment (HM2) - When the CMM Meets ISO 9001 . 542Robert C. Bamford, William J. Deibler II, Software Systems Quality ConsultingCD ROM Information . 554Index . 555Proceedings order form. back page

PrefaceSue Bartlett – PNSQCTerry Deupree – ASQ Software DivisionWelcome to the Sixteenth Annual Pacific Northwest Software Quality Conference and the EighthInternational Conference on Software Quality. We believe this conference is a superb example of howgreat things happen when two talented organizations get together on an idea. In the past year or so, wehave seen Windows NT starting to take over in areas once in the UNIX domain. We are also seeing anincreased demand in the marketplace for higher quality software in the shrink wrap market. Companiescontinue the quest for more productivity and less people. This makes our job as software professionals thatmuch more demanding to both produce quality software and to continuously search for better ways tosupport a more productive environment.This Conference offers an excellent opportunity to learn from successful practitioners and well-knownexperts. We are pleased to present these Proceedings, and the CD of the Proceedings, which contain the 32papers of the technical program. These papers were selected from over 60 abstracts submitted byprofessionals from all over the globe. In addition to the selected papers, we have invited seven excitingspeakers and two special panels to share their expertise and insights.We are delighted with the opportunity to bring Tom DeMarco as a keynote speaker this year. He has been amainstay of the software industry from his first book on structured analysis in 1978 which set a standard forprogramming quality. A personal favorite, Peopleware: Productive Projects and Teams brings to lightsome common sense ideas and some not-so-common sense research reports on how to give a person anopportunity to be more successful by paying attention to their environment. His latest book, The Deadline:A Novel about Project Management draws on his abundance of experience running projects, hard learnedover the years.On the second day we bring you Tom Gilb, the author of one of the first books on software metrics. Withyears on the front line, Tom has published additional books on software engineering management and hasbeen very active in software process debates. He will discuss the merits of the current software processpush in relation to his experience in the Software Quality arena. A veteran of the many “QualityInitiatives” over the years, he can provide us with insightful views of the current process wars.We would like to thank Theresa Hunt and Hilly Alexander, the Program Committee Chairs, for putting inthe hard work and many hours it takes to organize the program. Many thanks also to the ProgramCommittee members for their effort in selecting these papers from over 60 abstracts and their tirelessreviews of the draft papers. Additional thanks to Linda Westfall, from the ASQ Software Division, andSandy Raddue, from the PNSQC, who were priceless in their part of working out the agreements betweenthe two organizations and all the details that resulted.The members of all the committees have volunteered many hours and we thank them for their contributionsto the success of this Conference. Feel free to talk to any of the committee members as you see themwalking about (they wear ribbons marked “Committee”). We all welcome your suggestions.Finally, special thanks to our conference manager, Terri Moore of Pacific Agenda. She has handled theorganizational and administrative tasks very efficiently as always, and kept all the committees on track andon schedule.

CONFERENCE OFFICERS/COMMITTEE CHAIRSSue Bartlett - PNSQC President/ChairOrCAD, Inc.Mike Kress - Publicity Co-ChairBoeing Commercial Airplane GroupTerry Deupree - ASQ SoftwareDivision 8ICSQ ChairDSC CommunicationsTheresa Hunt - Program ChairECC InternationalSandy Raddue - PNSQC VicePresident, ASQ Software DivisionLiaisonCypress SemiconductorHilly Alexander - Program Co-ChairADP Dealer ServicesLaurie Duff - PNSQC SoftwareExcellence AwardADP Dealer ServicesLinda Westfall - ASQ SoftwareDivision Program ChairDSC CommunicationsShauna Gonzales - Birds of a FeatherImageBuilder SoftwareDennis Ganoe - PNSQC SecretaryWacom TechnologiesDon White - Exhibits ChairBidTekRay Lischner - PNSQC Treasurer,CD-Rom ProductionTempest SoftwareGeree Streun - Exhibits Co-ChairGV Software Solutions, Inc.Tom Griffin - CD-Rom ProductionAuburn UniversityRick Clements - Keynote Chair 1998,Publicity ChairFlir SystemsHoward Mercier - Workshop ChairStep TechnologyDoug Hamilton - Workshop Co-ChairAnderson Consulting

CONFERENCE PLANNING COMMITTEERick BiehlData-Oriented Quality SolutionsBill JunkUniversity of IdahoGreg BorchersSharp LaboratoriesRandy KingDan CampoDSC CommunicationsPam CaseCase Consulting ServicesKingsum ChowIntel Corp.Carolee CosgroveMcCain Foods Ltd.Jayash DalalLucent TechnologiesTaz DaughtreySoftware Quality ProfessionalClaire LohrLohr SystemsSue McGrathSAS Institute Inc.Patricia McQuaidCal Poly State UniversityFred MowlePurdue UniversityLynn MoyersPhoenix TechnologiesRusty PerkinsLockheed Martin Corp.John PustaverCindy EllisAdvanced Sterilization ProductsGalina GolantCypress SemiconductorEric SchnellmanBoeingGayla SuddarthTanning TechnologyBhushan GuptaDick HamletPortland State UniversityWarren HarrisonPortland State UniversityDan HoffmanUniversity of VictoriaCraig ThomasSequent Corp.Miguel UlloaIntelLynne WarrenMotorolaScott WhitmireAdvance Systems ResearchMark JohnsonOrCAD Corp.Rachel JordonCypress Semiconductor