Transcription
PM World JournalVol. III, Issue VI – June 2014www.pmworldjournal.netAgile & EVM: Expert Knowledge from EVM WorldCPM – Earned Value SeriesSeries Article1by Shobha MahabirSeries on Earned Value Management1Agile & EVM:Expert Knowledge from EVM WorldBy Shobha MahabirIntroductionA practice symposium track titled “AGILE EVM” was presented at EVM World 2013.The interest in the subject was evident by the large number of attendees, at some times 20% of the conference attendees. The track would not have been possible without thesupport of our presenters and their employers.Scott Dalessio, Grant ThorntonJoy L. Villagomez, Lockheed MartinJosh Breen. Grant ThorntonGlen Alleman, PrimePMPete Zafros, Grant ThorntonRon Terbush, Lockheed MartinThis article summarizes the six presentations within the track.BackgroundEarned Value Management has traditionally been applied to the waterfall type ofdevelopment with fixed scope and fixed technical requirements. The Agile methodologycontinuously analyzes both the scope and technical requirements to keep pace with thecustomer’s evolving environment. With Agile, the cost and schedule are fixed; thereforethe customer gets the most value achievable at a known cost and on a predetermineddate.Introduction to Agile & ScrumAgile is an alternative to the traditional and waterfall project managementmethodologies. The term Agile was introduced in the Agile Manifesto of 2001. 2 Agile isbased on iterative development used primarily in software development. It focuses on1This series of articles, some previously published in The Measurable News, is provided by the College ofPerformance Management (CPM), the world’s leading professional organization devoted to Earned ValueManagement and integrated project and program performance management. This article originally appeared inthe The Measureable News in 2013, republished here with approval of the author and CPM. More informationon CPM can be found at www.mycpm.org.2Agile Manifesto at http://www.agilemanifesto.org/ 2014 author namewww.pmworldlibrary.netPage 1 of 11
PM World JournalVol. III, Issue VI – June 2014www.pmworldjournal.netAgile & EVM: Expert Knowledge from EVM WorldCPM – Earned Value SeriesSeries Article1by Shobha Mahabiradaptive planning and evolutionary development that is rapid and flexible. The Agilemethodology is value - and vision - driven, and it has iterative cycles. It involves polyskilled teams which foster communication and make sure that they have the requireddocumentation for a project. Similarly, the Office of Management and Budget (OMB)focuses on “modular development”, which is referenced in OMB guidance onContracting Guidance to Support Modular Development.3Some aspects of using Agile that need to be understood include: Agile is NOT a silver bullet for software development problemsAgile does not mean no planning, processes, or documentationAgile is not appropriate for every type of projectAs Scott Dalessio, a presenter at EVM World 2013, stated, “Agile is a silver mirror notsilver bullet.” Zac Gery explains this as the foundation of Agile does not solve problems;it helps identify potential obstacles. All decisions whether good or bad are reflectedusing the methodology.4 Agile is not appropriate for every type of project. It is adaptiveand flexible; however, you need to determine what makes sense for the team.The results of the 7th Annual State of Agile Development Survey5 showed that Scrumwas used by over 50% of respondents; it is the most widely adopted agile method.Scrum is an iterative and incremental project delivery framework that relies onorganized, self-managing, and cross-functional teams. The Scrum roles include theproduct owner, the Scrum Master, and the team. The Scrum lifecycle is depicted inFigure 1 below.Figure 1: Scrum Lifecycle63OMB Contracting Guidance to Support Modular Development tion-technology.pdf 4Zac Gery,” Reflections in Scrum’s Silver Mirror”, 16 June 2012. ionsplusinplusScrum-27splusSilverplusMirror .5 th7 Annual State of Agile Development Survey: AgileDevelopment-Survey.pdf .6Source: Copyright 2008-2010 Mitch Lacey http://www.mitchlacey.com . 2014 author namewww.pmworldlibrary.netPage 2 of 11
PM World JournalVol. III, Issue VI – June 2014www.pmworldjournal.netAgile & EVM: Expert Knowledge from EVM WorldCPM – Earned Value SeriesSeries Article1by Shobha MahabirScrum meetings comprise of the following activities:Applying Elements of AgileEven if a project is not labelled “Agile”, applying a few elements of the methodologymay still be beneficial. The two programs presented at the conference by RonTerbush and Joy Villagomez illustrated some successes of evolving from waterfall toAgile methodology in selected process. These areas include management ofexpectations, scope, and risk with a system backlog; implementation of Just in Timeplanning and risk-based statusing; executing cycles of iterative development withcontinuous integration, verification, and validation; partnering EVM reporting withAgile burn-down charts; and enabling program success by being structured, yetflexible.During the iterative development cycle, focus is placed on results, not progression.Feedback from user testing sessions can be incorporated into the next cycle throughrapid feedback loops. At the iteration level, weekly and monthly percent completefeedback is done for EVM status. Weekly, on-demand feedback using iterationbacklog is provided on total percent complete and individual tasks hotspots thatprovide a quick way to identify, organize and scan what is important. At the cyclelevel, weekly feedback is provided on the watch list across iterations. This feedback isdriven by independent test assessment of requirements and user story satisfaction.The completion trend for the cycle is done at the work package level.It is important not to lose sight of the backlog and to take a step back to look at thedata at various levels. Key areas to focus on are the budget and cycle, together knownas iteration planning – how to map a waterfall budget to the Agile lifecycle. Just inTime planning methodology, where work is planned in near term to incorporate 2014 author namewww.pmworldlibrary.netPage 3 of 11
PM World JournalVol. III, Issue VI – June 2014www.pmworldjournal.netAgile & EVM: Expert Knowledge from EVM WorldCPM – Earned Value SeriesSeries Article1by Shobha Mahabirlessons learned from prior cycles, is critical. Continuous verification and validationoccurs during each iteration. Shared lessons learned include: Don’t wait to integrate. The team is responsible for continuous iteration. Don’t wait to verify. Test representatives on the team continuously checkrequirements-based functionality during the iteration via dry runs. Don’t wait to validate. The team works to continuously check useracceptability during the iteration. Don’t wait to measure verification/validation progress. The team continuouslymonitors iteration-level progress via iteration backlog percent done. Test orCMO independently monitors application-level progress via percent donethrough user stories and/or requirements satisfactionSome of the challenges with the initiation of Agile Development includedorganization culture, roles and responsibilities, staffing, and co-location. Having achampion for process improvements is crucial to the success of an Agile program.Members of the team stepped up as coaches, did the research, and coached theirteam members to understand the philosophy of Agile Scrum. Continuous learningthrough book clubs assisted with adhering to the process at first, the team then madeuse of retrospectives to reflect on efficiency. Sprint zero, a preliminary sprint focusedon preparing for the first sprint, is a necessity and upfront planning is essential.Short sprints mean more repetitions, which lead to more practice and greaterimprovement.Methods to Apply EVM to AgileTraditional EVM under the waterfall methodology requires up-front planning toimplement the requirements of a system. It determines budget and schedule for theentire scope of the work. Progress is tracked to the original budget and schedule.Traditional EVM is less flexible with limited ability to make changes throughout theprocess.Agile uses a more dynamic approach. It breaks down high-level features inincrements (sprints) after budget and schedule have been determined. This approachallows for continuous baseline revision to reflect the work completed (story points)during sprints. Progress is measured by making allowances for iterative changes.Adopting EVM to Agile projects uses Agile Scrum processes to produce EVM metrics.It utilizes planned accomplishments, called story points, as a basis for determiningEarned Value (EV); however, EV is not taken for partial completion. Story points areearned with the item and accepted by the users or customers. Therefore, the EVtechnique of 0/1oo percent is used. Frequent changes to the baseline are allowedwhen story points are added or removed, typically the end of each sprint. 2014 author namewww.pmworldlibrary.netPage 4 of 11
PM World JournalVol. III, Issue VI – June 2014www.pmworldjournal.netAgile & EVM: Expert Knowledge from EVM WorldCPM – Earned Value SeriesSeries Article1by Shobha MahabirAgile terms differ from the traditional EVM due to the integration of sprints andstory points. Budget at Complete (BAC) - the planned budget for the release or projectPlanned Percent Complete (PPC) – the number of current sprints divided bythe total number of planned sprintsActual Percent Complete (APC) – the total number of story points completeddivided by story points plannedThe most significant change is in the EVM calculations for Planned Value (PV) andEarned Value (EV). Planned Value (PV) [progress against planned sprint schedule] PPC x BACEarned Value (EV) [progress against planned budget] APC x BACNote that Planned Value is defined using sprints, while Earned Value isdefined using story points.Some leading practices shared by Josh Breen during the Agile EVM PracticeSymposium: The basis of EVM is an accurate baselineThe baseline in Agile EVM must be established at the beginning of a releaseProgress needs to be measured at defined boundariesThe initial data points are needed when planning the baseline:o Number of planned sprintso Length of the sprintso Number of story points planned for releaseo Budget planned for the releaseo Start and end datesThe planned work in a release is expressed in stories, which are quantifiedstory pointsAt the start of each sprint, stories are selected from the product backlog andserve as the baseline for the sprintStories may not be added or removed during the sprint, and no value is earnedfor partial completion of a storyIf an assigned story is not completed during a sprint, it is placed back into theproduct backlog to be baselined in a later sprintProgress and performance are measured at predefined boundariesAfter each iteration (sprint or release), the product backlog must be reviewed,and the future baseline must be adjusted to account for past performance.Integrating Agile with EVMWhether the methodology is traditional or Agile, the goal is to increase the probabilityof program success. ANSI-748B is the Earned Value Management standard. While the32 ANSI-748B criteria are designed to deliver value, touch points identified by GlenAlleman provide the basis for connecting the dots. 2014 author namewww.pmworldlibrary.netPage 5 of 11
PM World JournalVol. III, Issue VI – June 2014www.pmworldjournal.netANSICriteria #Agile & EVM: Expert Knowledge from EVM WorldCPM – Earned Value SeriesSeries Article1EVM Criteriaby Shobha MahabirAgile Approach1Define WBSFeatures and stories define tasks2Identify organizationSelf-organizing team5Integrate OBS and WBSSelf-organized teams with customer6Schedule workIterations and releases7Identify deliverables andmilestonesWorking software at the end of each iterations8Set time phased budgetFixed length iterations and releases16Record direct costsFixed staff level of effort23Determine variancesVelocity measures missed features25Sum data and varianceMissed features moved to next iteration26Manage action plansReplan missed features, adjust velocity28Incorporate changesReplan missed features, adjust velocityTable 1: Eleven Critical EVM Criteria with AgileThe Office of Management and Budget (OMB) announced in June 2012 the “ContractingGuidance to Support Modular IT development” that provides guidance to empoweragency leadership and Integrated Project Teams to support rapid delivery of workablesolutions. From the National Defense Acquisition Act (NDAA) section 804, it seems thatthe Department of Defense (DoD) is ready to embrace Agile methodologies as depictedin the view of DoD IT Acquisition Lifecycle’s Impact on Agile below.Figure 2: Business Capability Lifecycle (BCL) Acquisition Business Model7A New Approach for Delivering Information Technology Capabilities in theDepartment of Defense. OSD 13744 -10 804 Report to Congress.7 2014 author namewww.pmworldlibrary.netPage 6 of 11
PM World JournalVol. III, Issue VI – June 2014www.pmworldjournal.netAgile & EVM: Expert Knowledge from EVM WorldCPM – Earned Value SeriesSeries Article1by Shobha MahabirFederal Implementation and Support for AgileIn the GAO’s study on “Software Development: Effective Practices and FederalChallenges in Applying Agile Methods” 32 practices and approaches were identified aseffective for applying Agile software development methods to IT projects. 8 There wereten practices that were used and found effective by officials from the five federalagencies that were part of the study: Start with Agile guidance and an Agile adoption strategy. Enhance migration to Agile concepts using Agile terms, such as user stories (usedto convey requirements), and Agile examples, such as demonstrating how towrite a user story. Continuously improve Agile adoption at both the project and organizationallevel. Seek to identify and address impediments at the organization and project level. Obtain stakeholder/customer feedback frequently. Empower small, cross-functional teams. Include requirements related to security and progress monitoring in your queueof unfinished work (the backlog). Gain trust by demonstrating value at the end of each iteration. Track progress using tools and metrics. Track progress daily and visibly.Pete Zafros shared solutions to some of the challenges highlighted in the GAO’s study:ProcessScope &PlanningBudget &ProcurementChallengeSolution Scope statements are often toospecific to allow for innovation ornot focused enough to effectivelymanage a project Use business architectures, strategydocumentation, process analysis todefine the major functions of thesystems Scope is defined upfront and allrequirements must fit within thescope Understanding scope will minimizerework and ensure the system isbuilt in a sound fashion Upfront budgeting can force eitherwaterfall requirements processesor inflated budget requests Extended procurement processesAddress budgets iteratively or at theportfolio level to allow for improvedfluidity of funding between projectsUse precise procurements fordefined tasks 8Software Development: Effective Practices and Federal Challenges in Applying Agile Methods, GAO-12-681,Jul 27, 2012. www.gao.gov/assets/600/593091.pdf 2014 author namewww.pmworldlibrary.netPage 7 of 11
PM World JournalVol. III, Issue VI – June 2014www.pmworldjournal.netProcessSchedule & mentAgile & EVM: Expert Knowledge from EVM WorldCPM – Earned Value SeriesSeries Article1by Shobha MahabirChallengeSolutioncause the early on-boarding ofstaff which leads to periods of lowutilization and lack of optimizedefforts Seek extensive skill sets,communication skills and agileexperience for less defined tasksSchedule and cost estimation arethe most flawed aspects ofsoftware development and themost common reason forperceived project failures Attempt to make iterative estimatesto the greatest extent possible Establish consistency in scope andrequirements between projects tobuild historical data that can beused effectively A solid foundation of scope andbroad system requirements are nottypically established prior todevelopment causing rework Identify scope and broad strokeimplementation requirementsupfront Staff had difficulty identifyingwhich requirements/issues mustbe addressed immediately andthose that do are notBuild detailed requirementsaccording to business priority,common understanding andtechnical feasibility Requirements are not Yes or No,they are Now or Maybe Later Skill sets may be less broad thanthe optimal agile developer andlead to bigger teams than usual When teams are large focuscommunications around businesscontext rather than status updates Requirements are not alwaysbroken enough to allow fordevelopment while unknowns anddependencies exist Paired programming can preventdependencies on individuals,provide developers a broaderunderstanding of their efforts andbuild skill sets Having testers that are distinctfrom other project roles increasesthe team size and complexity ofcommunication Small user stories will drive testbased environments Requirements analysts, productowners and functional staff shouldbe actively involved in testing Incorporate mandatory deploymenttesting during development Identify external testing groups touse the product in testingenvironments if frequentdeployments are not possible Make sure deployment testingenvironments match productionenvironments Testers are not used to upfrontinteraction with developers Lengthy test plans and postdevelopment testing often causesprints to become a series of shortwaterfall cycles Lengthy and expensivedeployment processes forcewaterfall delivery and minimizethe collection of essential iterativefeedbackTable 2: Challenges and Recommended Solutions 2014 author namewww.pmworldlibrary.netPage 8 of 11
PM World JournalVol. III, Issue VI – June 2014www.pmworldjournal.netAgile & EVM: Expert Knowledge from EVM WorldCPM – Earned Value SeriesSeries Article1by Shobha MahabirPanel DiscussionThe presenters of the Agile EVM practice symposium were part of a paneldiscussion at the end of all the sessions in this track. Some of the questions that camefrom attendees in the room as well as from the virtual transmission are listed below:Question 1: How is performance taken?Panel Response: The story points are actually the work done or accomplished. At theend of a sprint if the capability is not delivered you will not be able to claim 100%complete. Therefore the work planned and not delivered may drive a cost varianceand definitely a schedule variance. The percent complete can be established againstthe dollar value of the story point to determine the Quantifiable Backup Data (QBD).Question 2: Explain how the story point becomes part of the calculation.Panel Response: For the story point to become part of the calculation and become aQBD the team needs an objective way to measure performance; the story pointsequals in-scope tasks. Each story point becomes an assessment of percent completeeither 0% or 100% complete. A task contains multiple story points that rolls up to thepercent complete of the task at the summary level.Question 3: What do you base the Basis of Estimate (BOE) on?Panel Response: The BOE is based on flexibility to meet the timeline. Documentassumptions to provide the ability to drive to the minimal viable product.Question 4: Where is the work package (WP) defined at the sprint orrelease level?Panel Response: The work package is defined at the higher level to encourage theAgile processes. A sprint is a time-boxed event that has content and scope. Therelease has defined scopeQuestion 5: Compare your risk management approach between theTraditional versus Agile methodology.Panel Response: Agile keeps the risk process in place and reduces the risks earlier on,therefore you may not have the need for as much management reserve as you wouldusing the traditional approach.Question 6: If story points are used to measure performance how is thecost per story point determined to establish the budget?Panel Response: One option would be to not create any cost estimates associatedwith story points. The other option is to work with the team to determine the numberof hours associated with the story point to estimate the dollar value. As the teamdevelops their velocity or units of work completed in a certain interval they can use 2014 author namewww.pmworldlibrary.netPage 9 of 11
PM World JournalVol. III, Issue VI – June 2014www.pmworldjournal.netAgile & EVM: Expert Knowledge from EVM WorldCPM – Earned Value SeriesSeries Article1by Shobha Mahabirthe velocity as a metric to track the actual costs in dollars or hours against theplanned costs.Question 7: Is an Integrated Baseline Review (IBR) on an Agile programhandled differently than it is on a traditi0nal waterfall program?Panel Response: It is primarily handled the same. From experience shared by onemember of the panel, the only sidebar was with the software developers on theirprocesses with the Earned Value Management System (EVMS) Director.ConclusionThere is a move to adopt Agile in federal agencies with the directives from OMB withregard to modular development and the updated DoD acquisition guidance addressingthe use of Agile methods. The Agile EVM practice symposiums provide an excellentresource to learn about the topics. Please visit the College of Performance ManagementEVM library for more information. 2014 author namewww.pmworldlibrary.netPage 10 of 11
PM World JournalVol. III, Issue VI – June 2014www.pmworldjournal.netAgile & EVM: Expert Knowledge from EVM WorldCPM – Earned Value SeriesSeries Article1by Shobha MahabirAbout the AuthorShobha MahabirUSAShobha Mahabir is a consultant with PwC one of the top consulting firms in theUSA. She served as a volunteer at EVM World 2013. Ms. Mahabir holds a BA inEconomics from the University of Maryland, College Park and a MS in InformationSystems Technology from the George Washington University. Ms. Mahabir is aCertified Project Management Professional and Earned Value Professional, andholds certifications in ITIL Foundations and Intermediate. She has contributed on thedevelopment of the GAO Cost Estimating and Assessment Guide. Ms. Mahabir hascomprehensive experience in technical and project management, possessing morethan 15 years of combined IT and project management related work experience. Herexperience includes earned value management, capital planning and investmentcontrol, scheduling, cost estimating and budgeting, resource planning, projectmanagement, contract administration, and design, operation, and maintenance ofdatabases. Ms. Mahabir has also implemented and delivered EVM on multiple largefederal IT programs. 2014 author namewww.pmworldlibrary.netPage 11 of 11
through user stories and/or requirements satisfaction Some of the challenges with the initiation of Agile Development included organization culture, roles and responsibilities, staffing, and co-location
