Transcription
Scrum and CMMI Level 5: The Magic Potion for Code WarriorsJeff Sutherland, Ph.D.Carsten Ruseng JakobsenKent JohnsonPatientkeeper Inc.Systematic Software EngineeringAgileDigm ent.johnson@agiledigm.comAbstractProjects combining agile methods with CMMI1 aremore successful in producing higher quality softwarethat more effectively meets customer needs at a fasterpace. Systematic Software Engineering works atCMMI level 5 and uses Lean Software Development asa driver for optimizing software processes. Early pilotprojects showed productivity on Scrum teams almosttwice that of traditional teams. Other projects using astory-based test-driven approach to softwaredevelopment reduced defects in final test by 40%.We assert that Scrum and CMMI together bring amore powerful combination of adaptability andpredictability than either one alone and suggest howother companies can combine them.1. IntroductionSuccessful software development is challenged bythe ability to manage complexity, technologyinnovation, and requirements change. Agile and CMMImethods each address these challenges but have a verydifferent approach and perspective.Management of complexity requires processdiscipline while management of change requiresadaptability. CMMI provides process discipline andScrum enhances adaptability. This paper provides ananalysis of the effect of introducing Agile practicesinto a CMMI Level 5 company.1.1. CMMIThe Capability Maturity Model (CMM) hasexisted since 1991, as a model based on best practicesfor software development. It describes an evolutionarymethod for improving an organization from one that isad hoc and immature to one that is disciplined andmature [1]. The CMM is internationally recognizedand was developed by the Software EngineeringInstitute at Carnegie Mellon University, Pittsburgh,USA.1 Capability Maturity Model, CMM and CMMI areregistered in the U.S. Patent and Trademark OfficeIn 2002, a significantly extended version calledCMMI was announced, where the ‘I’ stands for‘Integration.”. In 2006, version 1.2 of the CMMI modelwas published [2]. This model integrated softwareengineering, systems engineering disciplines, and softwareacquisition practices into one maturity model. CMMIdefines 25 process areas to implement. For each processarea required goals, expected practices, and recommendedsub-practices are defined. In addition, a set of genericpractices must be applied for all processes.Experience with CMM and CMMI, demonstrates thatorganizations appraised to higher levels of CMM orCMMI improve the ability to deliver on schedule, cost,and agreed quality. Increasingly, the industry requiressuppliers to be appraised to CMM or CMMI level 3 orhigher [3]. A number of governmental requirements. For example, the Danish Minister ofScience recently proposed regulations to require publicorganizations to request documentation of their supplier’smaturity level [4].1.2. ScrumScrum for software development teams began at EaselCorporation in 1993 [5] and emerged as a formal methodat OOPSLA’95 [6]. A development process was needed tosupport enterprise teams where visualization of designimmediately generated working code. Fundamentalproblems inherent in software development influenced theintroduction of Scrum: Uncertainty is inherent and inevitable in softwaredevelopment processes and products - Ziv’sUncertainty Principle [7] For a new software system the requirements will notbe completely known until after the users have used it- Humphrey’s Requirements Uncertainty Principle [8] It is not possible to completely specify an interactivesystem – Wegner’s Lemma [9] Ambiguous and changing requirements, combinedwith evolving tools and technologies makeimplementation strategies unpredictable.“All-at-Once” models of software developmentuniquely fit object-oriented implementation of softwareand help resolve these challenges. They assume the1
creation of software involves simultaneous work onrequirements, analysis, design, coding, and testing,then delivering the entire system all at once [10].Sutherland and Schwaber, co-creators of Scrumjoined forces with creators of other Agile processes in2001 to write the Agile Manifesto [11]. A commonfocus on working software, team interactions, customercollaboration, and adapting to change were agreedupon as central principles essential to optimizingsoftware productivity and quality.2. Guide for mixing CMMI and Agile2.1. How CMMI can improve AgileOur focus is on using CMMI to help an organizationinstitutionalize Agile Methods. Selective use of AgileMethods may degenerate into undisciplined hacking.We believe real value from Agile Methods can only beobtained through disciplined use. CMMI has a conceptof Institutionalization that can help establish thisneeded discipline.Institutionalization is defined in CMMI as “theingrained way of doing business that an organizationfollows routinely as part of its corporate culture.”Others have described institutionalization as simply“this is the way we do things around here.” Note thatinstitutionalization is an organizational-level conceptthat supports multiple projects.CMMI supports institutionalization throughGeneric Practices (GP) associated with all processareas. For the purposes of discussion, we will look atthe 12 generic practices associated with maturity levels2 and 3 in the CMMI [14] and how they might help anorganization use Agile Methods. We have paraphrasedthe generic practices (shown in bold text below) tomatch our recommended usage with Agile Methods.In CMMI terms, the projects in an organization wouldbe expected to perform an activity that accomplishedeach of these generic practices. We have used Scrumas the example Agile Method to describe some of theactivities that relate to these practices.2.1.1. Establish and maintain an organizationalpolicy for planning and performing Agile Methods(GP 2.1). The first step toward institutionalization ofAgile Methods is to establish how and when they willbe used in the organization. An organization mightdetermine that Agile Methods will be used on allprojects or some subset of projects based on size, typeof product, technology, or other factors. This policy isa way to clearly communicate the organization’s intentregarding Agile Methods. In keeping with the AgilePrinciple of face-to-face conversations at “all handsmeeting” or a visit by a senior manager during a project’skick off could be used to communicate the policy.2.1.2. Establish and maintain the plan for performingAgile Methods (GP2.2). This practice can help ensureAgile Methods do not degrade into undisciplined hacking.The expectation is that Agile Methods are planned andthat a defined process exists and is followed. The definedprocess should include a sequence of steps capturing theminimum essential information needed to describe what aproject really does. The plan would also capture theessential aspects of how the other 10 generic practices areto be implemented in the project. In Scrum, some of thisplanning is likely to be captured in a product backlogand/or sprint backlog, most likely within a tool as opposedto a document.2.1.3. Provide adequate resources for performing AgileMethods (GP 2.3). Every project wants, needs, andexpects competent professionals, adequate funding, andappropriate facilities and tools. Implementing an activityto explicitly manage these wants and needs has proveduseful. In Scrum, for example, these needs may bereviewed and addressed at the Sprint Planning Meetingand Sprint Review Meeting and reconsidered whensignificant changes occur.2.1.4. Assign responsibility and authority forperforming Agile Methods (GP 2.4). For a project to besuccessful, clear responsibility and authority need to bedefined. Usually this includes a combination of roledescriptions and assignments. The definitions of theseroles identify a level of responsibility and authority. Forexample, a Scrum Project would assign an individual orindividuals to the roles of Product Owner, ScrumMaster,and Team. Expertise in the Team is likely to include a mixof domain experts, system engineers, software engineers,architects, programmers, analysts, QA experts, testers, UIdesigners, etc. Scrum assigns the team as a whole theresponsibility for delivering working software. TheProduct Owner is responsible for specifying andprioritizing the work. The ScrumMaster is responsible forassuring the Scrum process is followed. Management isresponsible for providing the right expertise to the team.2.1.5. Train the people performing Agile Methods (GP2.5). The right training can increase the performance ofcompetent professionals and supports introducing newmethods into an organization. Institutionalization of theAgile Method being used requires consistent training. Thispractice includes determining the individuals to train,defining the exact training to provide, and performing theneeded training. Training can be provided using manydifferent approaches, including programmed instruction,formalized on-the-job training, mentoring, and formal andclassroom training. It is important that a mechanism be2
defined to ensure that training has occurred and isbeneficial.2.1.6. Place designated work products underappropriate level of configuration management (GP2.6). The purpose of a project is to produce deliverableproduct(s). This product is often a collection of anumber of intermediate or supporting work products(code, manuals, software systems, build files, etc.).Each of these work products has a value and often goesthrough a series of steps that increase their value. Theconcept of configuration management is intended toprotect these valuable work products by defining thelevel of control, for example, version control orbaseline control and perhaps multiple levels of baselinecontrol to use within the project.2.1.7. Identify and involve the relevant stakeholdersas planned (GP 2.7). Involving the customer as arelevant stakeholder is a strength of Agile Methods.This practice further identifies the need to ensure thatthe expected level of stakeholder involvement occurs.For example, if the project depends on customerfeedback with each increment, build, or Sprint, andinvolvement falls short of expectations, it is thennecessary to communicate to the appropriate level,individual, or group in the organization to allow forcorrective action, as corrective action may be beyondthe scope of the project team. In advanced Scrumimplementations, this may be formalized as aMetaScrum [17] where stakeholders serve as a boardof directors for the Product Owner.2.1.8. Monitor and control Agile Methods againstthe plan and take appropriate corrective action (GP2.8).This practice involves measuring actualperformance against the project’s plan and takingcorrective action. Direct day-to-day monitoring is astrong feature of the Daily Scrum Meeting, the ReleaseBurndown Chart shows how much work remains at thebeginning of each Sprint, and the Sprint BurndownChart shows total task hours remaining per day. Scrumenhances the effectiveness of the plan by allowing theProduct Owner to inspect and adapt to maximize ROI,rather than merely assuring plan accuracy.2.1.9. Objectively evaluate adherence to the AgileMethods and address noncompliance (GP2.9). Thispractice is based on having someone not directlyresponsible for managing or performing projectactivities evaluate the actual activities of the project.Some organizations implement this practice as both anassurance activity and coaching activity. The coachingconcept matches many Agile Methods. TheScrumMaster has primary responsibility for adherenceto Scrum practices, tracking progress, removingimpediments, resolving personnel problems, and isusually not engaged in implementation of project tasks.The Product Owner has primary responsibility for assuringsoftware meets requirements and is high quality.2.1.10. Review the activities, status, and results of theAgile Methods with higher-level management andresolve issues (GP2.10). The purpose of this practice is toensure that higher-level management has appropriatevisibility into the project activities. Different managershave different needs for information. Agile Methods havea high level of interaction, for example, Scrum has aSprint Planning Meeting, Daily Scrum Meetings, a SprintReview Meeting, and a Sprint Retrospective Meeting.Management needs are supported by transparency ofstatus data produced by the Scrum Burndown Chartcombined with defect data. Management responsibilitiesare to (1) provide strategic vision, business strategy, andresources, (2) remove impediments surfaced by Scrumteams that the teams cannot remove themselves, (3) ensuregrowth and career path of staff, and (4) challenge theScrum teams to move beyond mediocrity. The list ofimpediments generated by the Scrum teams is transparentto management and it is their responsibility to assure theyare removed in order to improve organizationalperformance.2.1.11. Establish and maintain the description of AgileMethods (GP 3.1). This practice is a refinement of GP2.2above. The only real difference is that description ofAgile Methods in this practice is expected to beorganization-wide and not unique to a project. The resultis that variability in how Agile Methods are performedwould be reduced across the organization; and thereforemore exchange between projects of people, tools,information and products can be supported.2.1.12. Collect the results from using Agile Methods tosupport future use and improve the organization’sapproach to Agile Methods (GP 3.2).This practice supports the goal of learning across projectsby collecting the results from individual projects. TheScrum Sprint Retrospective Meeting could be used as themechanism for this practice.All of these generic practices have been useful inorganizations implementing other processes. We haveseen that a number of these generic practices have at leastpartial support in Scrum or other Agile Methods. Webelieve that implementing these practices can helpestablish needed discipline to any Agile Method.2.2. Critiques of CMMIn research funded by the Danish government, Roseet. al. surveyed the literature on critiques of CMM [18].They observed that the chief criticism of CMM is not theprocess itself, but the effects of focus on processorientation.3
While side effects of process focus may be viewedas simply poor CMM implementation, organizationswith heavyweight processes are highly prone to poorexecution.As with any other model, good and badimplementations of CMM exist. We believe that badimplementations are one of the main reasons for theexistence of many negative criticisms of CMM. Suchimplementations are often characterized as in the tablebelow, whereas many good CMM implementationsaddress most of the criticism.One way to enhance chances for a good CMM orCMMI implementation is to use Scrum. ApplyingScrum and agile mindset while implementing CMMIwill help to assure good practice.We acknowledge that the CMM criticism listed inthe table below exist, but from our knowledge of goodCMMI implementations we consider the criticisms tobe incorrect. However, a poor implementation ofCMMI may be perceived this to have the problemsbelow. Even though good CMMI implementations canbe done without agile methods, the table shows thatScrum will contribute with a beneficial focus on issuesstemming from a “bad” CMMI implementation.Conversely, there are Agile implementations thatdo no meet the basic requirements of Scrum practice.They do not result in fixed iterations that deliverworking code that is fully tested at the end of eachSprint. The Product Owner may not be clearly definedor be dysfunctional. There may not be a single ProductBacklog prioritized by business value for the companyleading to conflicting priorities and thrashing withinthe organization. The Product Backlog may not beestimated by the team, leading to unpredictableprojects, missed deadlines, and broken budgets. Theteam may not keep a Burndown chart and know thevelocity of software production from Sprint to Sprint.This can make it impossible for the Product Owner todevelop a release plan with predicable release dates. Agood CMMI implementation can help remedy all ofthese common impediments.CMM criticismCMM reveres process butignores people.Scrum supportScrum is the firstdevelopment process totreat people issues thesame as other projectmanagement issues [19].CMMI does not focus onunderlying organizationalproblems that should beresolved.CMMI ignores quality inthe software productassuming an unprovenlink between quality in theprocess and quality in theresultingproduct.Differing project andorganizationalcircumstances may meanthat a process that deliversa good product in onecontext delivers a poorproductinanothercontext.CMMI lacks businessorientationCMMI provides poorawarenessoforganizational context.CMMI ignores technicalandorganizationalinfrastructures.CMMI encourages aninternal efficiency focusand thus market andcompetition blindness.A primary responsibilityof the ScrumMaster is tomaintain and resolve animpediment list thatcontains organizationalissues, personal issues,and technical problems.TheScrumProductOwner is responsible forcontinuouslyreprioritizing the ProductBacklog to maximizebusiness value in currentcontext.The primary focus ofScrum is on deliveringbusiness value.With Scrum, creation andprioritization of features,tasks, and impediments isalwaysdoneinorganizational context byinspected and adapting.Daily inspection andadaptation in Scrummeetings focuses ontechnicalandorganizational issues.Scrum focus is ondelivering business value.Type C Scrum allows ecting and adaptinginrealtimetocompetition [17].3. Scrum and CMMI: a magic potionSystematic is a privately held software systems companythat employs over 400 people with offices in Denmark,USA, Finland and the UK. They develop large systemsused in the defense, healthcare, manufacturing, and serviceindustries. In November 2005 they were appraised using4
the SCAMPISM2 method and found to be CMMI level 5compliant.At Systematic, CMMI Level 5 practices havereduced rework by 42%, maintained estimationprecision deviation less than 10%, and assure 92% ofall milestones are delivered early or on time. At thesame time, extra work on projects has beensignificantly 7%2%Q2 2005Q3 2005Q4 2005Q1 2006Q2 2006Q3 2006Q4 2006Q1 2007Figure 1 Rework at SystematicMore importantly, Systematic has transformedover twenty years of experience into a unified set ofprocesses used by all software projects. Historical dataare systematically collected and analyzed tocontinuously provide insight into the capability andperformance of the organization.The use of a shared common process makes iteasier for people to move from project to project andshare experiences and lessons learned betweenprojects. We can also compare performance of newprocesses to performance of existing processes andcreate a foundation for continuous improvement.using 69% effort compared to a CMMI Level 1 company[12, 13]. Figure 2 above shows that replacing some coreprocesses with Scrum drives cost down another 34%, cutsprocess overhead by more than 50% and drives defectsdown by 40%.CMMI Level 5 is increasingly a requirement fromcustomers and key to obtaining large contracts, especiallywithin defence and healthcare. Customers recognize thatCMMI Level 5 gives high predictability and betterengineered product for scalability, maintainability,adaptability, and reliability.CMMI provides insight into what processes areneeded to maintain a disciplined mature organizationcapable of predicting and improving performance of theorganization and projects. Scrum provides guidance forefficient management of projects in a way that allows forhigh flexibility and adaptability. When mixing the two, amagic potion emerges, wher
into a CMMI Level 5 company. 1.1. CMMI The Capability Maturity Model (CMM) has existed since 1991, as a model based on best practices for software development. It describes an evolutionary method for improving an organization from one t
