Transcription
Standard CMMISMAppraisal Method forProcess ImprovementSM(SCAMPI ),Version 1.1:Method DefinitionDocumentMembers of the Assessment Method Integrated TeamDecember 2001HANDBOOKCMU/SEI-2001-HB-001
Pittsburgh, PA 15213-3890Standard CMMISMAppraisal Method forProcess ImprovementSM(SCAMPI ),Version1.1:Method DefinitionDocumentCMU/SEI-2001-HB-001Members of the Assessment Method Integrated TeamDecember 2001CMM IntegrationSM ProjectUnlimited distribution subject to the copyright.
This report was prepared for theSEI Joint Program OfficeHQ ESC/DIB5 Eglin StreetHanscom AFB, MA 01731-2116The ideas and findings in this report should not be construed as an official DoD position. It is published in the interest ofscientific and technical information exchange.FOR THE COMMANDERNorton L. Compton, Lt Col, USAFSEI Joint Program OfficeThis work is sponsored by the U.S. Department of Defense. The Software Engineering Institute is afederally funded research and development center sponsored by the U.S. Department of Defense.Copyright 2001 by Carnegie Mellon University.NO WARRANTYTHIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL ISFURNISHED ON AN "AS-IS" BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANYKIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO,WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINEDFROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OFANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT.Use of any trademarks in this report is not intended in any way to infringe on the rights of the trademark holder.Internal use. Permission to reproduce this document and to prepare derivative works from this document for internal use isgranted, provided the copyright and "No Warranty" statements are included with all reproductions and derivative works.External use. Requests for permission to reproduce this document or prepare derivative works of this document for externaland commercial use should be addressed to the SEI Licensing Agent.This work was created in the performance of Federal Government Contract Number F19628-00-C-0003 with CarnegieMellon University for the operation of the Software Engineering Institute, a federally funded research and developmentcenter. The Government of the United States has a royalty-free government-purpose license to use, duplicate, or disclose thework, in whole or in part and in any manner, and to have or permit others to do so, for government purposes pursuant to thecopyright license under the clause at 252.227-7013.For information about purchasing paper copies of SEI reports, please visit the publications portion of our Web ml).printed 1/14/2002 12:16 PM 2.0 PDC
Table of ContentsAbstractPart I: OverviewxiI-1About This Document3Document Outline3Audiences for This Document5How to Use This Document5Feedback Information7Executive SummaryWhat is SCAMPI?Core Concepts and ApproachSCAMPI Method Overview991015Method Context15Method Objectives and Characteristics15Modes of Usage16Method Performance18Method Concepts20Method Assumptions and Design Principles 20Verification vs. DiscoveryCMU/SEI-2001-HB-00122I-i
Focused Investigation23Practice Implementation Indicators24Data Collection, Rating, and Reporting26Instruments and Tools29Effective Team Practices31Method Description32Part II: Process DefinitionsII-21.1 Analyze RequirementsII-41.1.1 Determine Appraisal ObjectivesII-71.1.2 Determine Appraisal ConstraintsII-91.1.3 Determine Appraisal ScopeII-111.1.4 Determine OutputsII-151.2 Develop Appraisal PlanII-201.2.1 Tailor MethodII-231.2.2 Identify Needed ResourcesII-251.2.3 Determine Cost and ScheduleII-271.2.4 Plan and Manage LogisticsII-291.2.5 Document and Manage RisksII-301.2.6 Obtain Commitment to AppraisalPlanII-311.3 Select and Prepare Team1.3.1 Identify Team LeaderI-iiII-34II-37CMU/SEI-2001-HB-001
1.3.2 Select Team Members1.3.3 Prepare Team1.4 Obtain and Analyze Initial ObjectiveEvidenceII-41II-481.4.1 Prepare ParticipantsII-511.4.2 Administer InstrumentsII-531.4.3 Obtain Initial Objective EvidenceII-551.4.4 Inventory Objective EvidenceII-571.5 Prepare for Collection of ObjectiveEvidenceII-601.5.1 Perform Readiness ReviewII-631.5.2 Prepare Data Collection PlanII-651.5.3 Replan Data CollectionII-692.1 Examine Objective EvidenceII-722.1.1 Examine Objective Evidence fromInstrumentsII-752.1.2 Examine Objective Evidence fromPresentationsII-772.1.3 Examine Objective Evidence fromDocumentsII-792.1.4 Examine Objective Evidence fromInterviewsII-812.2 Verify and Validate Objective EvidenceII-862.2.1 Verify Objective EvidenceCMU/SEI-2001-HB-001II-39II-89I-iii
2.2.2 Characterize Implementation ofModel PracticesII-912.2.3 Validate Practice ImplementationGapsII-952.3 Document Objective Evidence2.3.1 Take/Review/Tag NotesII-1032.3.2 Record Presence/Absence ofObjective EvidenceII-1052.3.3 Document PracticeImplementation GapsII-1072.3.4 Review and Update the DataCollection PlanII-1092.4 Generate Appraisal ResultsII-1122.4.1 Derive Findings and Rate GoalsII-1152.4.2a Determine Process AreaCapability LevelII-1172.4.2b Determine Satisfaction ofProcess AreasII-1192.4.3a Determine Capability ProfileII-1202.4.3b Determine Maturity LevelII-1212.4.4 Document Appraisal ResultsII-1223.1 Deliver Appraisal ResultsI-ivII-100II-1243.1.1 Present Final FindingsII-1273.1.2 Conduct Executive Session(s)II-1313.1.3 Plan for Next StepsII-133CMU/SEI-2001-HB-001
3.2 Package and Archive Appraisal Assets II-136CMU/SEI-2001-HB-0013.2.1 Collect Lessons LearnedII-1393.2.2 Generate Appraisal RecordII-1413.2.3 Provide Appraisal Feedback toCMMI StewardII-1433.2.4 Archive and/or Dispose ofKey ArtifactsII-144Part III: Appendices, References, andGlossaryIII-1Appendix A: Appraisal DisclosureStatementIII-3Appendix B: The Role of PracticeImplementation Indicators in VerifyingPractice ImplementationIII-5Appendix C: Focused InvestigationElaboration and GuidanceIII-13Appendix D: ARC/MDD Traceability I-31I-v
I-viCMU/SEI-2001-HB-001
List of FiguresCMU/SEI-2001-HB-001Figure I-1: Example of PII UseI-26Figure I-2: SCAMPI Rating ProcessI-27Figure I-3: Process Flows, Plan and PrepareProcessesI-38Figure I-4: Process Flows, Conduct AppraisalProcessesI-39Figure I-5: Process Flows, Report ResultsProcessesI-40Figure I-6: Nominal Schedule for AssessmentModeI-41Figure I-7: Nominal Schedule for EvaluationModeI-42I-vii
I-viiiCMU/SEI-2001-HB-001
List of TablesTable I-1: Part I ContentsI-3Table I-2: Part II ContentsI-4Table I-3: Part III ContentsI-4Table I-4: Process Definition ElementsI-6Table I-5: Activity Description ElementsI-7Table I-6: Essential Characteristics of theSCAMPI MethodI-16Table I-7: SCAMPI Modes of UsageI-17Table I-8: Potential Targets for AppraisalPerformance ImprovementI-19Table I-9: Practice Implementation IndicatorTypesI-25Table I-10: SCAMPI Phase Summary: Plan andPrepare for AppraisalI-35Table I-11: SCAMPI Phase Summary: ConductAppraisalI-36CMU/SEI-2001-HB-001Table I-12: SCAMPI Phase Summary: ReportResultsI-37Table III-1: PII TypesIII-8Table III-2: A PIID SchemaIII-10Table III-3: An Example PIIDIII-10Table III-4: ARC TraceabilityIII-15I-ix
I-xCMU/SEI-2001-HB-001
AbstractThe Standard CMMI Appraisal Method for Process Improvement (SCAMPISM) is designed toprovide benchmark quality ratings relative to Capability Maturity Model Integration(CMMISM) models. It is applicable to a wide range of appraisal usage modes, including bothinternal process improvement and external capability determinations. SCAMPI satisfies all ofthe Appraisal Requirements for CMMI (ARC) requirements for a Class A appraisal methodand can support the conduct of ISO/IEC 15504 assessments.The SCAMPI Method Definition Document describes the requirements, activities, and practices associated with each of the processes that compose the SCAMPI method. It is intendedto be one of the elements of the infrastructure within which SCAMPI Lead Appraisers conduct a SCAMPI appraisal. Precise listings of required practices, parameters, and variationlimits, as well as optional practices and guidance for enacting the method, are covered. Anoverview of the method’s context, concepts, and architecture is also provided. Capability Maturity Model is registered in the U.S. Patent and Trademark Office.SCAMPI, CMMI, and CMM Integration are service marks of Carnegie Mellon University.SMCMU/SEI-2001-HB-001I-xi
I-xiiCMU/SEI-2001-HB-001
Part I: OverviewCMU/SEI-2001-HB-001I-1
I-2CMU/SEI-2001-HB-001
About This DocumentThe Method Definition Document (MDD) describes the Standard CMMISM Appraisal Methodfor Process Improvement (SCAMPISM). It is intended to meet the needs of different readers.The document is divided into three major parts, each with a different level of detail, intendedusage, and primary audience. The structure, audiences, and suggested use of each part of thedocument are described below.Document OutlinePart I: OverviewThis part of the document provides an overview of the method’s context, concepts, and architecture. The reader is provided with the big picture of the method, rather than details abouthow to enact it. Table I-1 shows the contents of Part I.Table I-1: Part I ContentsSectionPagesAbout This DocumentI-3 – I-7Executive SummaryI-9 – I-13SCAMPI Method OverviewI-15 – I-41Part II: Process DefinitionsThis part of the document provides the definitive coverage of method requirements and detailed activities and practices associated with each of the processes that compose theSCAMPI method. Precise listings of required practices, parameters, and limits of variationallowable, as well as optional practices and guidance for enacting the method, are covered inthis core part of the document. Table I-2 shows the contents of Part II.SMCMMI, CMM Integration, and SCAMPI are service marks of Carnegie Mellon University.CMU/SEI-2001-HB-001I-3
Table I-2: Part II ContentsPhaseProcessPages1: Plan and Preparefor Appraisal1.1 Analyze RequirementsII-3 – II-171.2 Develop Appraisal PlanII-19 – II-311.3 Select and Prepare TeamII-33 – II-451.4 Obtain and Analyze Initial ObjectiveEvidenceII-47 – II-581.5 Prepare for Collection of Objective EvidenceII-59 – II-692.1 Examine Objective EvidenceII-71 – II-842.2 Verify and Validate Objective EvidenceII-85 – II-972.3 Document Objective EvidenceII-99 – II-1092.4 Generate Appraisal ResultsII-111 – II-1213.1 Deliver Appraisal ResultsII-123 – II-1333.2 Package and Archive Appraisal AssetsII-135 – II-1432: Conduct Appraisal3: Report ResultsPart III: AppendicesThe material contained in the appendices of this document provide further elaboration on selected topics, and are intended to supplement the material in the first two parts of the document. Rarely will someone who has not already read the first two parts read an appendix ofthis document. The topical elaboration and reference material available in the appendiceshelp to provide deeper insight to an already knowledgeable reader. Table I-3 shows the contents of Part III.Table I-3: Part III ContentsSectionPagesAppendix A: Appraisal Disclosure StatementIII-3Appendix B: The Role of Practice Implementation Indicators in VerifyingPractice ImplementationIII-5 – III-11Appendix C: Focused Investigation Elaboration and GuidanceIII-13 – III-14Appendix D: ARC/MDD Traceability TableIII-15 – III-28ReferencesIII-29 – III-30GlossaryIII-31 – III-39I-4CMU/SEI-2001-HB-001
Audiences for This DocumentThe MDD is primarily intended for SCAMPI Lead Appraisers authorized by the SoftwareEngineering Institute (SEI). It is expected that these professionals meet prerequisites forknowledge and skills specified by the SEI Appraiser program (see http://www.sei.cmu.edu/for details), and that this document is one of the elements of the infrastructure within whichthey operate. They are considered the primary audience for Part II. Candidate Lead Appraisers will also use the MDD while attending training to learn about the method.Appraisal team members (under the leadership of an authorized SCAMPI Lead Appraiser)are expected to refer to this document as a training aid. Portions of the document may also beused as work aids during the conduct of an appraisal. Potential appraisal team members canuse the MDD to build their knowledge base for future participation in an appraisal.Finally, the larger set of stakeholders for the conduct of any given appraisal are also in thetargeted audience for the document, particularly for Part I. These stakeholders include: appraisal sponsors—leaders who sponsor appraisals to meet business needs Process Group members—process improvement specialists who need to understand themethod, and perhaps help others to gain familiarity other interested parties who wish to have deeper insight into the methodology for purposes such as ensuring that they have an informed basis for interpreting SCAMPI outputsor making comparisons among similar methodologiesHow to Use This DocumentPart IIt is expected that every member of the audience for this document will find value in Part I.The two primary sections in this part are the Executive Summary and the Method Overview.The Executive Summary is intended to provide high-level information about what SCAMPIis, and does not require extensive knowledge of appraisals. This portion of the document maybe excerpted and provided to a more casual reader or a stakeholder in need of general information to support their decision to conduct an appraisal.The Method Overview section provides more comprehensive coverage of SCAMPI, and canbe used to begin building a base of knowledge for readers who have need of more detailedinformation. Appraisal sponsors wanting more than the summary view described above willwant to read this section. Every prospective SCAMPI team leader and team member is expected to read this section of the document, to ensure that they have the big picture beforestudy of the detailed methodology begins.CMU/SEI-2001-HB-001I-5
Part IIPeople who will enact an appraisal are expected to read the second part of the document.Members of this audience need to know how to enact the method, not just what the methodis. Part II is divided into Process Definitions, which are in turn divided into Activity Descriptions. Each Activity Description delineates Required Practices, Parameters and Limits, Optional Practices, and Implementation Guidance.There are eleven processes contained in SCAMPI. The processes (as defined) support a variety of orderings and enactments to facilitate a variety of usage modes for SCAMPI. The temporal flow, as well as the flow of inputs and outputs among the processes, is described in theMethod Overview section. The Process Definitions are not intended to provide a start-tofinish view of SCAMPI. Rather, these sections provide detailed definitions of processes andactivities that are invoked according to the appraisal plan created by the appraisal teamleader.Each of the Process Definitions begins with a three-page overview of the process. Everyprocess is defined by information contained in the elements shown in Table I-4.Table I-4: Process Definition ElementsElementDescriptionPurposeA brief summary of what is accomplished by enacting the processEntry CriteriaConditions that must be met before enacting the processInputsArtifacts or information needed to enact the processActivitiesThe set of actions which, in combination, make up the processOutputsArtifacts and assets that result from enacting the processOutcomeAny change in important conditions or artifacts that results from enacting the processExit CriteriaConditions to be met before the process can be considered completeKey PointsA summary of the most notable events associated with the processTools andTechniquesWork aids commonly used in enacting the processMetricsUseful measures that support the process enactment, or future enactmentsVerification andValidationTechniques to verify and/or validate the enactment of the processRecordsInformation to be retained for future useTailoringBrief discussion of key tailoring options (not an exhaustive list)Interfaces withOther ProcessesDiscussion of how the process interacts with other processes in themethodSummary ofActivitiesA narrative summary of the set of activitiesI-6CMU/SEI-2001-HB-001
Following the three pages of introductory material, each activity that is a part of the processis briefly summarized to orient the reader to the scope of the activity. Each Activity Description includes the elements shown in Table I-5.Table I-5: Activity Description ElementsElementDescriptionRequired PracticesA listing of practices that must be implemented toconsider the enactment a valid SCAMPIParameters and LimitsAcceptable limits for things that are allowed to vary,and acceptable limits for things under the discretion ofthe appraisal team leaderOptional PracticesActions that reflect good practice but are not requiredImplementation GuidanceNarrative description of advice or things to consider inperforming the activityComplete and unambiguous descriptions of the method processes and activities are providedin this part of the document. In combination with the training materials and work aids thatcompose the CMMI Steward’s appraiser program, this information provides a firm basis forstandardization (within reasonable limits) of the practice of Process Appraisals.Part IIIThe appendices of the document provide detailed coverage of special topics as well as reference material. Readers knowledgeable in SCAMPI are expected to read these sections forfurther understanding.Feedback InformationWe are very interested in your ideas for improving this document. See the CMMI Web sitefor information on how to provide feedback: ts.html .If you have questions, send an email to cmmi-comments@sei.cmu.edu.CMU/SEI-2001-HB-001I-7
I-8CMU/SEI-2001-HB-001
Executive SummaryWhat is SCAMPI?The Standard CMMI Appraisal Method for Process Improvement (SCAMPI) is designed toprovide benchmark quality ratings relative to Capability Maturity Model Integration(CMMISM) models. It is applicable to a wide range of appraisal usage modes, including bothinternal process improvement and external capability determinations. SCAMPI satisfies all ofthe Appraisal Requirements for CMMI (ARC) requirements for a Class A appraisal methodand can support the conduct of ISO/IEC 15504 assessments.SCAMPI V1.1 enables a sponsor to gain insight into an organization’s engineering capability by identifying the strengths andweaknesses of its current processes relate these strengths and weaknesses to the CMMI model prioritize improvement plans focus on improvements (correct weaknesses that generate risks) that are most beneficialto the organization given its current level of organizational maturity or process capabilities derive capability level ratings as well as a maturity level rating identify development/acquisition risks relative to capability/maturity determinationsAs a Class A appraisal method, SCAMPI is an appropriate tool for benchmarking. Sponsorswho want to compare an organization’s process improvement achievements with other organizations in the industry may have a maturity level determined as part of the appraisalprocess. Decisions made on the basis of maturity level ratings are only valid if the ratings arebased on known criteria. Consequently, contextual information—organizational scope,CMMI model scope, appraisal method type, the identity of the Lead Appraiser and theteam—are items for which criteria and guidance are provided within the method to ensure aconsistent interpretation within the community. Benchmarking can only be valid when thereis a consistent basis for establishing the benchmarks. The SEI maintains industry aggregatesfor appraisal results. These data a
The Standard CMMI Appraisal Method for Process Improvement (SCAMPISM) is designed to provide benchmark quality ratings relative to Capability Maturity Model Integration (CMMISM) models. It is applicable to a wide range of appraisal usage modes, including both internal process improvement and external capability determinations.
