WIXLIB

One framework for incremental development is Agile softwaredevelopment, which has been adopted by many federal agencies. Itemphasizes early and continuous software delivery and is defined byvalues and principles that can be realized through a set of commonpractices seen in specific Agile frameworks, such as DevOps, eXtremeProgramming, Lean, Kanban, Scrum, and others. 4This guide has been developed with the assistance of manyknowledgeable specialists in the field of Agile and other incrementalsoftware development methods to aid federal agencies, departments, andauditors in assessing an organization’s readiness to adopt Agilemethods. 5The best practices in this guide are not comprehensive; that is, they arepresented as high-level concepts of software development, contracting,and program management that highlight the aspects of Agiledevelopment throughout a program’s life cycle and address key risks toan organization, program, or team without prescriptive “how to” steps.Many other publications address how to apply best practices in using anincremental approach to software development and readers can refer tothose sources when considering a specific development topic.GAO, the Congressional Budget Office (CBO) and other organizationshave developed projections that show the nation’s fiscal path isunsustainable. New resource demands and demographic trends will placeserious budgetary pressures on federal discretionary spending, as well asother federal policies and programs in the coming years. When resourcesare scarce, competition for those resources increases. It is imperative,therefore, that government programs deliver the promised results, not4Agileframeworks are also used to develop hardware programs and manage services.The best practices in this guide are intended to be at a high enough level to be used forany Incremental development program, regardless what type of product or service is beingdelivered. However, the focus of this guide will be how Agile frameworks are used insoftware development.5Agileis the name we use to describe incremental software development methods in thisguide, with concepts from Lean, Kanban, DevOps, or other more specific methods. Forexample, Kanban may not be considered an Agile software development methodology,but it may be considered a management method used to improve the flexibility of theactivities of knowledge workers during software development. An organization that intendsto adopt a specific Agile method should supplement guidance described later in this guidewith additional materials that specifically address the practical application of that specificmethod.Page 2GAO-20-590G GAO Agile Assessment Guide

only because of their value to the public, but also because every dollarspent on one program is one less dollar available to fund other efforts.GAO plans to periodically update this exposure draft based on users’experience and comments. Comments and suggestions from experiencedusers and knowledgeable specialists regarding the application of Agileprinciples are welcome. Please click on this linkhttps://tell.gao.gov/agileguide to provide comments on this guide.If you have any questions concerning this guide, contact Tim Persons at(202) 512-6888 or personst@gao.gov or Carol Harris at (202) 512-4456or harriscc@gao.gov. Major contributors to this guide are listed inappendix IX and contact points for our Offices of Congressional Relationsand Public Affairs are located at the end of this document.Timothy M. Persons, Ph.D.Chief Scientist and ManagingDirectorScience, Technology Assessment,and Analytics TeamCarol HarrisDirectorInformation Technologyand Cybersecurity TeamPage 3GAO-20-590G GAO Agile Assessment Guide

IntroductionThe federal government spends at least 90 billion annually oninformation technology (IT) investments. In our January 2019 High RiskList report, GAO reported on 35 high risk areas, including themanagement of IT acquisitions and operations. 6 While the executivebranch has undertaken numerous initiatives to help agencies bettermanage their IT investments, these programs frequently fail or incur costoverruns and schedule slippages while contributing little to missionrelated outcomes. GAO has found that OMB continues to demonstrate itsleadership commitment by issuing guidance for covered departments andagencies to implement statutory provisions commonly referred to asFITARA. 7 However, application of FITARA at federal agencies has notbeen fully implemented. For example, as we stated in the 2019 High Riskreport, none of the 24 major federal agencies had IT managementpolicies that fully addressed the roles of their Chief Information Officers(CIO) consistent with federal laws and guidance. 8This Agile Guide is intended to address generally accepted best practicesfor Agile adoption, execution, and control. In this guide, we use the termbest practice to be consistent with the use of the term in GAO’s series ofbest practices guides. 9Developing the GuideOur approach to developing this guide was to ascertain best practices forAgile software development from leading practitioners and to developstandard criteria to determine the extent to which agency softwaredevelopment programs meet these practices. These best practices center6GAO,High Risk Series: Substantial Efforts Needed to Achieve Greater Progress on HighRisk Areas, GAO-19-157SP (Washington, D.C.: Mar. 6, 2019).7Theprovisions apply to the agencies covered by the Chief Financial Officers Act of 1990,31 U.S.C. § 901(b). These agencies are the Departments of Agriculture, Commerce,Defense, Education, Energy, Health and Human Services, Homeland Security, Housingand Urban Development, Justice, Labor, State, the Interior, the Treasury, Transportation,and Veterans Affairs; the Environmental Protection Agency, General ServicesAdministration, National Aeronautics and Space Administration, Nation ScienceFoundation, Nuclear Regulatory Commission, Office of Personnel Management, SmallBusiness Administration, Social Security Administration and the U.S. Agency forInternational Development. However, FITARA has generally limited application to theDepartment of Defense.8GAO-19-157SP.9GAO,GAO Cost Estimating and Assessment Guide: Best Practices for Developing andManaging Program Costs, GAO-20-195G (Washington, D.C.: Mar. 12, 2020), GAOSchedule Assessment Guide: Best Practices for Project Schedules, GAO-16-89G(Washington, D.C.: Dec. 22, 2015) and GAO Technology Readiness Assessment Guide:Best Practices for Evaluating the Readiness of Technology for Use in AcquisitionPrograms and Projects, GAO-20-48G (Washington, D.C.: Jan 7, 2020).Page 4GAO-20-590G GAO Agile Assessment Guide

on Agile adoption, execution, and control. We developed each bestpractice in consultation with a committee of IT and program managementspecialists and organization executives across government, privateindustry, and academia. We describe our scope and methodology indetail in appendix I.The Guide’s ReadersThe Guide’s ContentsWe have developed this guide to serve multiple audiences: The primary audience for this guide is federal auditors. Specifically,the guide presents best practices that can be used to assess theextent to which an agency has adopted and implemented Agilemethods. Organizations and programs that have already established policiesand protocols for Agile adoption and execution can use this guide toevaluate their existing approach to Agile software development. Organizations and programs that are in the midst of adopting Agilesoftware development practices and programs that are planning toadopt such practices can also use this guide to inform theirtransitions.This guide focuses on best practices surrounding Agile adoption,execution, and controls. For example, chapter 3 groups commonlyrecognized best practices for Agile adoption into the areas of teamdynamics and activities, program operations, and organizationenvironment. Chapter 4 provides an overview of high-level programmanagement concepts surrounding Agile execution and control bestpractices, such as requirements development and management,acquisition strategies, and program monitoring and control. Agileexecution best practices related to requirements development andmanagement and the federal contracting process are discussed in moredetail in chapters 5 and 6, respectively. Program control and monitoringbest practices for cost estimating, scheduling, and earned valuemanagement are discussed in chapter 7, and best practices for metricsthat can be used during the adoption, execution, and monitoring andcontrol periods of the program are discussed in chapter 8.Certain concepts in the chapters are further explained in the appendixes.Definitions of the key terms and processes discussed throughout thisguide are explained in appendix II and related terms that compare termswith similar meanings from different methodologies are displayed inappendix III.Page 5GAO-20-590G GAO Agile Assessment Guide

This guide also contains a number of case studies drawn from prior GAOwork. The case studies highlight successes and challenges typicallyassociated with Agile adoption and execution in federal settings. Thesecase studies are meant to augment the key points and the lessonslearned that each chapter discusses. For example, GAO has found thatproblems can arise due to the misapplication of Agile softwaredevelopment processes and methods. 10 Similar to the case studies, Agilein Action segments were developed by interviewing agency officials,reviewing documentation, and performing site visits to observe Agilemethods in use. To help verify that the information presented in theseexamples was complete, accurate, and up-to-date, we provided eachorganization with a draft version of our summary analysis. Appendix VIIprovides high-level information for each program used in a case studyand a summary of the Agile in Action process.AcknowledgmentsThe GAO Agile Assessment Guide team thanks the many subject matterexperts in the federal government, private industry, and academia whohelped make this guide a reality. After we discussed our initial plans fordeveloping this guide with GAO’s Cost Working Group and at varioustechnical conferences, several members expressed interest in workingwith us. They formed the initial membership of our Agile working groupthat convened in August 2016. This number grew as the work developed,and the contributions of all have been invaluable. Thanks to everyonewho gave their time and expertise in meetings, provided us withdocumentation and comments, and hosted us at their facilities as weobserved their Agile methods in real time. Contributors of the Agileworking group are listed in appendix VIII and GAO staff who contributedto this guide are listed in appendix IX.10Forexample, in GAO, Immigration Benefits System: US Citizenship and ImmigrationServices Can Improve Program Management (GAO-16-467) we reported that the UnitedStates Citizenship and Immigration Service Transformation program was not settingoutcomes for Agile software development and in TSA Modernization: Use of SoundProgram Management and Oversight Practices is Needed to Avoid Repeating PastProblems (GAO-18-46) we reported that the Transportation Security Administration’sTechnology Infrastructure Modernization (TIM) program did not define key Agile roles,prioritize system requirements, or implement automated capabilities.Page 6GAO-20-590G GAO Agile Assessment Guide

Chapter 1: BackgroundChapter 1: BackgroundThe most well-known feature of Agile software development is probablyits emphasis on iterative product development and delivery; that is,development of software in iterations that are being continuouslyevaluated on their functionality, quality, and customer satisfaction. 11 Thismethod is well suited for programs in which the final product is to includedistinct features, some of which may be discovered during the processrather than planned at the beginning. Information obtained during thesefrequent iterations can effectively assist in measuring progress andallowing developers to respond quickly to feedback from customers, thusreducing technical and programmatic risk. 12 With its emphasis on earlyand continuous delivery of working software, Agile can be a valuable toolfor organizations in helping to mitigate schedule and budget risks.Figure 1 compares requirements, design, development, and testing usingAgile software methods to those of the formerly-used Waterfallframework; it illustrates how requirements, design, development, andtesting are performed concurrently in small iterations for Agile andsequentially in Waterfall development. 13 In contrast to Waterfall, using anAgile framework can result in an organization producing software usingfrequent reviews and customer feedback to help ensure that the highestvalue requirements are being met. Figure 1 compares Agile and Waterfallmethods for developing software, assuming that high-level planning forboth Agile and Waterfall development has already occurred.11In this guide, an iteration is a predefined, time boxed, recurring period of time in whichworking software is created. Similarly, a release is defined as a planned segment ofrequirements that are useable. For more information, see appendix II.12The term ‘customer’ can mean different things depending on the perspective. Forexample, often a customer refers to the end users of a system but there are alsoinstances where the customer and sponsor are the same individual. The definition of thecustomer(s) is organizationally and contextually dependent. See appendix II for moreinformation on how we define this term and use it throughout the guide.13A 1970 paper entitled “Managing the Development of Large Software Systems” by Dr.Winston W. Royce is considered by the Software Engineering Institute and others to bethe basis for the Waterfall framework. (See Royce, Winston, “Managing the Developmentof Large Software Systems. Reprinted from proceedings, IEEE WESCOM (August 1970),pages 1-9). Although the paper never uses the term “Waterfall,” the model has sequentialphases that flow continuously from one step to the next. While the paper noted that thismodel is risky because it is unknown how the system will actually work until the testingphase and recommended iterative interaction between steps, it became the foundation forwhat is known as the Waterfall approach.Page 7GAO-20-590G GAO Agile Assessment Guide

Chapter 1: BackgroundFigure 1: Comparison of Agile and Waterfall Methods for Developing SoftwarePage 8GAO-20-590G GAO Agile Assessment Guide

Chapter 1: BackgroundThe Value of Using AgileWith an emphasis on the early and continuous delivery of working software, Agile canbe a valuable tool for mitigating risks by engaging customers in collaboration early in theprogram and continuously adapting to changing requirements and environment, thuslimiting the chance of continuing to fund a failing program or outdated technology.While some versions of incremental development were being used asearly as the 1950’s, the Agile approach was articulated in 2001 by agroup of 17 software developers that called themselves the Agile Alliance.In February 2001, the Alliance released “The Agile Manifesto,” in whichthey declared: “We are uncovering better ways of developing software bydoing it and helping others do it. Through this work we have come tovalue: Individuals and interactions over processes and tools. Working software over comprehensive documentation. Customer collaboration over contract negotiation. Response to change over following a plan.” 14The Alliance added that, while they recognized the value in the secondpart of each statement (e.g., “processes and tools”), they saw more valuein the first part (e.g., “individuals and interactions”). The Alliance furtherdelineated their vision with 12 principles. The 12 Agile principles behindthe Manifesto are:1. Our highest priority is to satisfy the customer through early andcontinuous delivery of valuable software.2. Welcome changing requirements, even late in development. Agileprocesses harness change for the customer’s competitive advantage.3. Deliver working software frequently, from a couple of weeks to acouple of months, with a preference to the shorter time scale.4. Business people and developers must work together daily throughoutthe project.5. Build projects around motivated individuals. Give them theenvironment and support they need, and trust them to get the jobdone.14 2001-2020Page 9Agile Manifesto authors https://agilemanifesto.org.GAO-20-590G GAO Agile Assessment Guide

Chapter 1: Background6. The most efficient and effective method of conveying information toand within a development team is face-to-face conversation.7. Working software is the primary measure of progress.8. Agile processes promote sustainable development. The sponsors,developers, and users should be able to maintain a constant paceindefinitely.9. Continuous attention to technical excellence and good designenhances agility.10. Simplicity, the art of maximizing the amount of work not done, isessential.11. The best architectures, requirements, and designs emerge from selforganizing teams.12. At regular intervals, the team reflects on how to become moreeffective, then tunes and adjusts its behavior accordingly. 15There are numerous approaches, or frameworks, available for Agileprograms to use. A framework is a basic structure to guide customers,rather than a prescriptive process. Each framework is unique and mayhave its own terminology for processes and artifacts, though theframeworks are not mutually exclusive and so can be combined. 16 Whenimplementing Agile in a federal environment, all staff, both governmentand contractor, will want to work together to define the Agile terms andprocesses that will be used for their particular program. Agile, as aconcept, is not prescriptive; however, when applied to an organization, itmay be. Regardless of the specific frameworks and practices, it isimportant that applying Agile aligns with the manifesto and Agileprinciples. A detailed description of commonly-used Agile frameworks islocated in appendix V. Table 1 provides a high-level definition for severalcommonly-used incremental development frameworks.15 2001-2020Agile Manifesto authors https://agilemanifesto.org.16Some frameworks vary from the Agile Manifesto’s values and principles. For example,Kanban must have a customer who has requested a service and an end point where therequest is fulfilled and delivered to the customer. In this case, the manifesto’s value of

AGILE ASSESSMENT GUIDE Best Practices for Agile Adoption and Implementation September 2020 GAO-20-590G From September 28, 2020 to September 27, 2021, GAO is seeking input and feedback on this Exposure Draft from