Transcription
Early Adoption Experienceswith CMMI for Acquisition(CMMI-ACQ)by Jon L Gross and Matt FisherSMCMM Integration, SCAMPI, SCAMPI Lead Appraiser, TSP, and IDEAL are service marks of CarnegieMellon University. CMMI, Capability Maturity Model, CMM, and Carnegie Mellon are registered in the U.S. Patent andTrademark Office by Carnegie Mellon University.For more information on CMU/SEI Trademark use, please rademarks-usage.htmlSponsored by the U.S. Department of Defense 2008 by Carnegie Mellon UniversityThis material is approved for public release.Early Adoption Experiences with CMMI for Acquisition 081508
Early Adoption Experiences with CMMI for AcquisitionOutline of Presentation Process improvement needs in an acquisition organization High level summary of CMMI for Acquisition (CMMI-ACQ) anddifferences with CMMI for Development (CMMI-DEV) Description of early adoption experiences and risks identified Teaching Appraisals Process Improvement ConsultingMitigation of early adoption risks 081508 2008 Carnegie Mellon University2
Early Adoption Experiences with CMMI for AcquisitionOutline of Presentation Process improvement needs in an acquisition organization High level summary of CMMI-ACQ and differences with CMMIDEV Description of early adoption experiences and risks identified Teaching Appraisals Process Improvement ConsultingMitigation of early adoption risks 081508 2008 Carnegie Mellon University3
Early Adoption Experiences with CMMI for AcquisitionSome definitions:Acquisition - “The process of obtaining products or servicesthrough a supplier agreement.” This would also include,outsourcing and/or supply chain; where supplieragreements are established.Supplier agreement - “A documented agreement betweenthe acquirer and supplier (e.g., contract, license, ormemorandum of agreement).”Source: CMMI-ACQ V1.2 modelNote: The laws and regulations of the country and organization are expected to befollowed when applying CMMI-ACQ in improved processes.081508 2008 Carnegie Mellon University4
Early Adoption Experiences with CMMI for AcquisitionWhat We Mean By AcquisitionAcquisition PhasesInitiationRFIContractAwardTIMsRFP haseAwardPhasePost-Award Phase. Acquisition planning Contract administration Bidders conference Monitoring technical performance RFP preparation Technical Interchange Meetings Solicitation Test and evaluation of deliverables Proposal evaluation Best and final offers Source selectionThese are all potentials areas for acquisition work process improvement.081508 2008 Carnegie Mellon University5
Early Adoption Experiences with CMMI for AcquisitionLifecycles Overlay – A complex environmentLifecycle Milestones0 Product Acquisition MsRFP eliverablesInitiationRFIN 1ContractAwardTIMsRFP ProposalsPre-AwardPhaseAwardPhasePost-Award Phase. Acquisition planning Contract administration Bidders conference Monitoring technical performance RFP preparation Technical Interchange Meetings Solicitation Test and evaluation of deliverables Proposal evaluation Best and final offers Source mpletion.Pre-AwardPhaseContractAwardTIMsRFP ProposalsNPre-AwardPhaseAwardPhasePost-Award Phase. Acquisition planning Contract administration Bidders conference Monitoring technical performance RFP preparation Technical Interchange Meetings Solicitation Test and evaluation of deliverables Proposal evaluation Best and final offers Source selectionPost-Award Phase. Acquisition planning Contract administration Bidders conference Monitoring technical performance RFP preparation Technical Interchange Meetings Solicitation Test and evaluation of deliverables Proposal evaluation Best and final offers Source selectionThese all potentials areas for acquisition work process improvement.081508 2008 Carnegie Mellon University6
Early Adoption Experiences with CMMI for AcquisitionA Survey of Software 028%23%49%199826%28%46%199627%40%33%Average cost growth exceeds 89%Average final product contains 61%of the originally specified featuresSoftware projectscompleted on time(Success)Projects canceled beforecompletion (Failed)Late and over budget(Challenged)What role did acquisition20% 40%60% 80% 100% organization play in results?0%We do not know, notRef: CHAOS Study, Standish Group,1994 - 2006measured.199416%31%53%081508 2008 Carnegie Mellon University7
Early Adoption Experiences with CMMI for AcquisitionThe State of Acquisition PracticeIs There an Acquisition Crisis?Investigation of one acquisition program showed:— System complexity and the program’s lack of experience inprocuring major systems caused serious cost growth.— Program lacked systems engineering and program managementexpertise.— Requirements stabilization process was absent.— Program management does not enforce timely milestones,timelines, and deliverables.— Program’s lack of process control made assessment of technicalrisk impossible.— Program’s lack of short- and long-term budget tracking made costassessment nearly impossible.— Program did not manage risk.Source: GAO Report081508 2008 Carnegie Mellon University8
Early Adoption Experiences with CMMI for AcquisitionAcquirer/Supplier MismatchAcquirerHighMismatchMatchedmature acquirer mentorslow maturity supplieracquirer and supplierare both high maturityoutcome notpredictablehighest probability ofsuccessMismatchLowDisasterTechnical &Management Skillno disciplineno processno productLowless mature acquirerderails mature supplier;encourages short cutssupplier compromisesprocessesSupplierHigh081508 2008 Carnegie Mellon University9
Early Adoption Experiences with CMMI for AcquisitionAddressing Mismatch - Why Look in theMirror?“He who has the gold makes the rules” and pays theprice.” All acquisitions begins inside the organization needing adevelopment or an outsourcing. Acquisition work processes lead to supplier agreements.Insanity -- “ doing the same thing over and over and expectingdifferent results. "-Benjamin Franklin Agencies have spent decades using magnifying glasses tolook at the performance of the suppliers/contractors. We still have problems with program success.It is time to start looking in a mirror081508 2008 Carnegie Mellon University10
Early Adoption Experiences with CMMI for AcquisitionAcquisition ManagementWHO IS IN CHARGE?And who is eventuality accountableAcquisitionproject teamAcquisition activitiesdeal with:The overall projectperformance,schedule, costSupplier activitiesdeal with:Supplierproject teamThe Supplier’sperformance,schedule, cost081508 2008 Carnegie Mellon University11
Early Adoption Experiences with CMMI for AcquisitionWhat Can Be Done? Based on the premise that“The quality of a system or product is highly influenced by thequality of the process used to develop or maintain it.”(CMMI-ACQ V1.2)We could improve the process and practices of the Supplier But the developers have a head start (CMMI-DEV basedimprovements are widespread) Has not solved all the problems We could improve the processes and practices of the Acquirer by: increasing the visibility of the Acquirer’s contribution to programsuccess defining, implementing, measuring and evolving effective acquisitionprocesses and practices081508 2008 Carnegie Mellon University12
Early Adoption Experiences with CMMI for AcquisitionAs the Acquirer, how would you like to work?Processes can make the difference.081508 2008 Carnegie Mellon University13
Early Adoption Experiences with CMMI for AcquisitionIneffective Approach to Quality AcquisitionHistorically, many acquisition organizations have approachedquality by performing product (contractor or supplier) measurementonly. But, this approach does not improve quality nor guaranteequality. It’s a strategy of defect detection, not defect prevention.Contractor/ SupplierProducts and ProcessesAcquisitionProcessesExamining with a‘magnifying glass’Measurement of final product attributes is too late.the quality, good or bad, is already in the product.100% inspection is only 80% effective — Dr. Joseph Juran081508 2008 Carnegie Mellon University14
Early Adoption Experiences with CMMI for AcquisitionExpected Acquisition Organization Result?Most acquisitionorganization do notmeasure their workprocesses.Most acquisitionorganizations tend to beoptimistic in the programprojections.Benefits of measuring: Better estimation Less deviation fromtargetAs you move up in maturity level, projectcost will decrease and distribution willbecome tighter and more predictableNotional for acquisition processes.081508 2008 Carnegie Mellon University15
Early Adoption Experiences with CMMI for AcquisitionOutline of Presentation Process improvement needs in an acquisition organization High level summary of CMMI-ACQ and differences withCMMI-DEV Description of early adoption experiences and risks identified —Teaching—Appraisals—Process Improvement ConsultingMitigation of early adoption risks081508 2008 Carnegie Mellon University16
Early Adoption Experiences with CMMI for AcquisitionEvolution of CMMI Models*SW-CMMV1.11993CMMI V1.1CMMI-DEV V1.2March 2002August 2006CMMI-SVCTBDEIA 7311998CMMI-AMV1.1SA-CMMMay 2005AdaptingCMMI for AcqJun 2006CMMI-ACQV1.2Nov e For Acquisition Improvement HasEvolved For Over a Decade.*Notional Time Scale081508 2008 Carnegie Mellon University17
Early Adoption Experiences with CMMI for AcquisitionTraditional Acquisition ActivitiesThe Supplier Agreementor on081508 2008 Carnegie Mellon University18
Early Adoption Experiences with CMMI for AcquisitionComparisons of CMMI-ACQ, V1.2 andCMMI-DEV, V1.2CMMI-ACQCMMI-DEVFor the acquirer (i.e., those who acquire, procure, orotherwise select and purchase products and services forbusiness purposes, or those who outsource developmentand support)For the product and service developer (i.e., those whodevelop or maintain products and services for businesspurposes)Focus on the acquisition of products and servicesFocus on the development and maintenance of productsand servicesGeneric practices are covered only in the Generic Goalsand Generic Practices sectionGeneric practices are covered both in the Generic Goalsand Generic Practices section and at the end of eachprocess areaExplicit coverage of servicesImplicit coverage of services through the definition of theterm “product,” which covers both products and servicesContains an Acquisition process area category, but noEngineering categoryContains an Engineering process area category, but noAcquisition categoryStages Acquisition Requirements Development atmaturity level 2Stages Requirements Development at maturity level 3Categorizes Requirements Management as a ProjectManagement process areaCategorizes Requirements Management as anEngineering process areaContains typical work products and typical supplierdeliverablesContains typical work -comparetoDEV.html081508 2008 Carnegie Mellon University19
Early Adoption Experiences with CMMI for AcquisitionCMMI-ACQ V1.2Continuous Representation: PAs by AcquisitionSupportProcess AreasOrganizational Process Focus Organizational Process Definition This chart shows all PAsin CMMI-ACQ. All PAs inthe Acquisition Categoryare unique to CMMI-ACQ.Organizational TrainingOrganizational Process PerformanceOrganizational Innovation and Deployment Project Planning Project Monitoring and Control Integrated Project Management Requirements ManagementRisk ManagementQuantitative Project Management Acquisition Requirements Development Acquisition Technical Management Acquisition Verification Acquisition Validation Agreement Management Solicitation and Supplier Agreement Development The other PAs with blueand surrounded by contain additionalspecific practices that areonly in the CMMI-ACQmodel.RequirementsManagement is in theProject Managementcategory in CMMI-ACQ.Configuration ManagementProcess and Product Quality AssuranceMeasurement and AnalysisDecision Analysis and ResolutionCausal Analysis and 12-comparetoDEV.html081508 2008 Carnegie Mellon University20
Early Adoption Experiences with CMMI for AcquisitionAcquirer/Supplier MismatchAcquirerHighMismatchMatchedmature acquirer mentorslow maturity supplieracquirer and supplierare both high maturityoutcome notpredictablehighest probability ofsuccessMismatchLowDisasterTechnical &Management Skillno disciplineno processno productLowless mature acquirerderails mature supplier;encourages short cutssupplier compromisesprocessesSupplierHigh081508 2008 Carnegie Mellon University21
Early Adoption Experiences with CMMI for AcquisitionAcquirer is in ChargeOperational andBusiness NeedCMMI for AcquisitionAcquirer urceSelectionProgram LeadershipInsight / OversightSystemAcceptanceTransitionTask OrdersSupplier ProcessesDo This AfterBasic ContractAwardedPlanDesignDevelopIntegrate& TestDeliverDeveloper or Supplier Processes(CMMI for Development based process improvement.)Operational and BusinessNeed Satisfied081508 2008 Carnegie Mellon University22
Early Adoption Experiences with CMMI for AcquisitionToday’s Approach to Quality Acquisition A strategy of problem prevention: focus on the acquisition workprocesses. Fixing the “acquisition processes” lowers the probability badproducts or services will be produced in the future acquisitions andsupplier agreements.The Supplier Agreementor ContractExaminingself with a‘mirror’AcquisitionWork ProcessesContractor/ SupplierProducts and Processes081508 2008 Carnegie Mellon University23
Early Adoption Experiences with CMMI for AcquisitionOutline of Presentation Process improvement needs in an acquisition organization High level summary of CMMI-Acquisition and differences withCMMI-DEV Description of early adoption experiences and risksidentified Teaching Appraisals Process Improvement ConsultingMitigation of early adoption risks 081508 2008 Carnegie Mellon University24
Early Adoption Experiences with CMMI for AcquisitionEarly Experiences from Teaching - 1Learning to Look in a Mirror is Tough “It is the suppliers problem, that is why wehired them.”A few acquisition organizations are willing tofocus on their internal improvement. Some US Government Agencies Some Foreign Government Agencies Some commercial companies081508 2008 Carnegie Mellon University25
Early Adoption Experiences with CMMI for AcquisitionEarly Experiences from Teaching - 2 They are using experiences from CMMI-DEVprocess improvement and applying them toacquisition work processes. Still holding the supplier accountable forcontract performance.081508 2008 Carnegie Mellon University26
Early Adoption Experiences with CMMI for AcquisitionEarly CMMI-ACQ Appraisal Experiences - 1General Motors performed SCAMPI A, B and C Appraisals,using process areas from CMMI-DEV and the Adapting CMMIfor Acquisition Organizations. SCAMPI Method and Classes work, without change Depends on acquisition organization defining their workprocessesGaining continuing Senior Level Sponsorship is Key, but can bepolitically sensitive. External pressure can be essential (e.g., GAO, Congress,CEO) Strategic View is important “I need to improve”A few US Government acquisition organizations are doing modelbased process improvement.081508 2008 Carnegie Mellon University27
Early Adoption Experiences with CMMI for AcquisitionEarly CMMI-ACQ Appraisal Experiences - 2The SCAMPI Appraisal Family of methods works for CMMI-ACQ,in the same way it works for CMMI-DEV.breadth of tailoringSCAMPI FamilySCAMPI C: provides a wide range of options,including characterization of planned approaches toprocess implementation according to a scale definedby the userCSCAMPI B: provides options in model scope andorganizational scope, but characterization ofpractices is fixed to one scale and is performed onimplemented practicesBSCAMPI A: Is the most rigorous method, and is theonly method that can result in ratings.depth ofinvestigationASCAMPI Lead Appraiser must be authorized for CMMI-ACQ.081508 2008 Carnegie Mellon University28
Early Adoption Experiences with CMMI for AcquisitionEarly CMMI-ACQ Appraisal Experiences - 3SCAMPI Appraisal Approach for CMMI-ACQ: Assure the appraisal team members have experienceand understanding of acquisition work. Assure model understanding by the organizational unitin preparing for appraisal.SCAMPI Class B and C appraisals are the best starting point— Uncovers model issues in a less intense appraisal environment— Allows time to align work processes with appraisal objectiveevidence expectations.— Appraisal Team Members need to complete SEIAuthorized Introduction to CMMI-ACQ V1.2 Training.081508 2008 Carnegie Mellon University29
Early Adoption Experiences with CMMI for AcquisitionEarly CMMI-ACQ Process ConsultingExperiences Transition from current process improvement model to CMMI-ACQ One Government organization is developing a transition process,with entry and exit criteria A commercial company is transitioning from acquisitionprocesses based on CMMI-DEV to new processes based onCMMI-ACQ Some organizations are starting with blank sheets of paper Lack of knowledge of model detail by senior management Sponsors often do not have enough detailed knowledge of themodels to understand the significance, benefits, or effort required Budget and/or senior management changes often affect processimprovement programs first081508 2008 Carnegie Mellon University30
Early Adoption Experiences with CMMI for AcquisitionOutline of Presentation Process improvement needs in an acquisition organization High level summary of CMMI-ACQ and differences withCMMI-DEV Description of early adoption experiences and risks identified —Teaching—Appraisals—Process Improvement ConsultingMitigation of early adoption risks081508 2008 Carnegie Mellon University31
Early Adoption Experiences with CMMI for AcquisitionThe Bottom Line or Mission Æ Must DriveImprovementImprovement means different things to different organizations: What are your mission or business goals? How do you measure progress?Improvement is a long-term, strategic effort: What is the expected impact on the mission or bottomline? How will impact be measured?081508 2008 Carnegie Mellon University32
Early Adoption Experiences with CMMI for AcquisitionGetting Started: Key Success Factors for anEffective PI EffortBefore major change can even be contemplated, topmanagement must recognize the need and be sold on the idea.The Must Haves:1.A compelling reason for change2.Leadership of the change effort by the top executive in theorganization — responsibility cannot be delegated3.Informed commitment of the top management team4.Designation of a primary PI Leader and an adequatemandate for change5.Sound performance measures that drive changeSource: Mastering Process Improvement Course081508 2008 Carnegie Mellon University33
Early Adoption Experiences with CMMI for AcquisitionGetting There Means CHANGING!Acquisition organizationshave many barriers toaddress to reach theculture of measuredprocess performance.Improvement of acquisition work processes is still in the“Innovator” or “Earlier Adopter” stage of transition.081508 2008 Carnegie Mellon University34
Early Adoption Experiences with CMMI for AcquisitionWhy So Long for Process Improvement?Levels of Learning RequiredCultureMost AcquisitionOrganizations areLarge.StrategyStructureProceduresDo you have longterm sponsor
CMM Integration, SCAMPI, SCAMPI Lead Appraiser, TSP, and IDEAL are service marks of Carnegie Mellon University. CMMI, Capability Maturity Model, CMM, and Carnegie Mellon are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University. For more information on CMU/SEI Trademark use, please visit .
