Transcription
Proceedings of the 52nd Hawaii International Conference on System Sciences 2019Team Autonomy in Large-Scale AgileNils Brede MoeSINTEFnils.b.moe@sintef.noBjørn DahlNTNUbjornhd@stud.ntnu.noLina Sund KarlsenNTNUlinaska@stud.ntnu.noAbstractLarge-scale software development is increasinglymaking use of agile practices. In large-scale projects, ateam needs to align with other teams and the rest of theorganization. This has been shown to threaten teamautonomy, which, in turn, reduces responsiveness andflexibility. Hence, agile teams face challenges inadapting to larger-scale development. We conduct amultiple case study of three large-scale projects toinvestigate barriers to team autonomy in large-scaleagile. Two barriers are identified: overall direction andexternal dependencies. We found that goals are often setby management without involving the teams, that theyare often equal to deliverables and deadlines, and thatteam members often do not know what the goals are.Consequently, teams struggle with setting andcommunicating goals as well as establishing a shareddirection. Organizational dependencies lead to teamshaving to deal with additional tasks, resulting in specificmembers shielding the teams from external noise.1. IntroductionLarge software development projects areincreasingly adopting agile development practices.Teams in large-scale projects need to reach agreementon many decisions with experts, managers, stakeholdersand other teams [28]. Further, quality concerns and theneed for frequent and coordinated releases forcescompanies to govern, control, and standardize multiteam development efforts. Therefore, the agile teamworking in a large-scale environment needs to bealigned with other teams and the rest of the organization.If the team breaks the quality or functionality or is late,it will affect other teams and deliverables. However, theneed for aligning the work, processes, and technologyand for coordinating externally is a threat to teamautonomy, which is the key to agility.URI: https://hdl.handle.net/10125/60136ISBN: 978-0-9981331-2-6(CC BY-NC-ND 4.0)Viktoria StrayUniversity of Oslo, SINTEFstray@ifi.uio.noStine Schjødt-OsmoNTNUstinsc@stud.ntnu.noThe notion of autonomy and self- management is notnew; research in this area has been around since Tristand Bamforth’s study of self-regulated coal miners inthe 1950s [34]. We use the label “self-managing teams”as a synonym for “autonomous teams,” and for“empowered teams.” Guzzo and Dickson [11] describesuch teams asemployees that typically perform highly related orinterdependent jobs, who are identified andidentifiable as a social unit in an organization, andwho are given significant authority andresponsibility for many aspects of their work, suchas planning, scheduling and assigning tasks tomembers, and making decisions with economicconsequence.While autonomous agile teams promise to increaseemployee motivation and job satisfaction significantly[15], as well as boost creativity and productivity [9],implementing such teams in a large-scale context ischallenging. When many agile teams are workingtoward the same goal, a lot of coordination andmanagement effort is required [7], and the team cannothave full authority over all aspects of the work as asingle one-project team. Further, in large innovativeprojects, the degree of complexity and uncertainty ishigh, as the work executed in teams is influenced by thework and inputs from other teams. While there is a needfor alignment and coordinated decision-making, Tataand Prasad [33] claim that team members need to affectmanagerial decisions genuinely in order to benefit fromself-management. Otherwise, they will experience onlysymbolic self-management, and if the managerialdecisions are only affected by symbolic input, the teammembers might hesitate to embrace self-management.Furthermore, for autonomous agile teams to worktogether in a large-scale project, there is a need fororganizational control and alignment, for the teams to beable to collaborate toward achieving the desiredobjective. Therefore, a single team cannot be entirelyautonomous in a large-scale environment.Page 6997
A question is, then, how to align teams withoutreducing team autonomy? What team externaldependencies and managerial decision hinders teamautonomy in large-scale agile? What reduces teamauthority in large-scale agile projects?Motivated by the importance of team autonomy inagile software development and the need for alignment,the main goal of this paper is to understand the enablersand barriers of autonomy, and to explore the conflictbetween autonomy on a team level and the need fororganizational control in large-scale agile softwaredevelopment. Our research question is:What are the barriers to team autonomy in largescale agile?In this paper, we examine team autonomy in thecontext of large-scale agile software development. Weunderstand large-scale development as a developmenteffort with many teams—from 3 to 20 teams [21].The remainder of the paper is organized as follows:In Section 2, we present background information onautonomous agile teams and large-scale agile. In section3, we describe our research method in detail. In Section4, we present results from an inductive multiple casestudy of three distinct large-scale projects across threecases. We discuss our findings in Section 5. Section 6concludes and presents key findings from the study.2. Autonomy in large-scale agileIn this section, we present background informationon autonomous agile teams and challenges inimplementing them. Second, we describe how agileteams are coordinated in large-scale agile.2.1. Autonomous agile teams and their barriersAgile teams usually consist of many—notnecessarily rigid—roles [32]. Back and Anders describethe roles typically found in an agile softwaredevelopment team [1]: developer/programmer, ager/product owner. To make all of these roles worktogether in a team, autonomous agile teams often makeuse of certain team practices or ceremonies. Stand-upmeetings, which are short, daily meetings in which teammembers share their work progress and possibleimpediments, are used to keep track of the progress ofthe software [30]. Retrospectives are another popularpractice. These are meetings in which the team membersreflect on past work processes: what we did well, whatwe want to keep doing, and what we want to do more of[18]. During a retrospective, the team members discusspossible measures that can ultimately improve thesustainability of the team. Furthermore, doingretrospectives frequently is associated with the businessvalue in the long run (ibid).According to Moe, Dingsøyr, and Dybå [22],autonomous agile teams should be responsible forplanning and scheduling their work, as letting theindividuals participate in these activities will increasetheir commitment to the team plan. Scrum and Kanbanare examples of bottom-up self-determined workdesigns that Parker and Wall [25] consider a definingfeature of autonomous teams. Stand-up meetings andretrospectives are also practices well within the aspectof control and management in the definition given of anautonomous team by Goodman, Devadas, and Hughson[10]. In other words, as Moe et al. [22] allude to, theresearch itself is not new, it has just found a new area ofapplication.Understanding how to enable autonomous softwaredevelopment teams requires more than just examiningthe team’s inner workings. We must also understand thebarriers at the team and organizational levels. In aninternational workshop on autonomous agile teams,Stray et al. [29] revealed the top barriers to be not havingclear and common goals, lack of trust, too manydependencies to others, lack of coaching andorganizational support, and diversity in team norms.Further, Moe et al. [22] identify several team-levelchallenges in a case study of a single agile team:individual commitment, failure to learn, and individualleadership. Individual commitment is linked to a lack ofcommitment to the team goals; they found that teammembers tended to pursue their own individual goalsinstead of the team goals. Failure to learn concernsprocess improvement; even though the team membersfrequently discussed potential changes, they did notimplement them. One reason was that the managementdid not set aside time for process improvement. Moe etal. (ibid) claim that if a team is not given the possibilityto improve, it will experience only symbolic selfmanagement, as explained by Tata and Prasad [33].Additionally, Stray, Moe, and Dingsøyr [31] foundthat even though agile methods were implemented,critical decisions were, in some cases, made by theproject managers without involving the developers.These findings are supported by Moe et al. [22] whofound that even though the concept of shared leadershipwas introduced to the teams in their study, teammembers did not change their decision-makingprocesses. This behavior led to difficulties in aligningdecisions when team members did not know what otherswere doing. Important decisions were also made withoutinforming the rest of the team, which led to a low levelof trust. For autonomous agile teams to be successful,Moe, Aurum, and Dybå [20] argue that team membersPage 6998
need to identify important decisions they should maketogether to be able to make the shared decisions they aresupposed to.Implementation of autonomous teams is difficult ifthere are barriers on the organizational level. One ofthese barriers is organizational control. Moe et al. [22]found that certain forms of detailed control by themanagement inhibit autonomy because the whole pointis that the teams should control themselves. Boehm andTurner [2] argue that this is where the project managercomes in; one of the project manager’s primary roles isto be the barrier between the organization and the team,preventing unnecessary interruptions.The two other challenges on the organizational levelare shared resources and specialist culture [22]. Sharedresources entail that projects fighting for resources andthe most skilled employees rarely build redundancy. Inother words, sharing resources across several projectsthreatens the autonomy. Specialist culture is a result oforganizations supporting and incentivizing being thebest at what one does rather than creating generalistswho can fill each other’s functions (ibid).2.2. Teams in Large-scale agileCoordinating externally is an issue for autonomousagile teams in large-scale agile. No team possesses allthe knowledge needed to solve complex tasks.Therefore, teams need to coordinate work with otherteams and experts. Further, as teams learn, productsbecome more mature, and the development processchanges, coordination mechanisms in large-scale agilechange [21]. According to Boehm and Ross [4], theprimary problem with project coordination is thatstakeholders such as users, customers, the developmentteam, and the management have to be simultaneouslysatisfied. This view is supported by Pikkarainen,Haikara, Salo, Abrahamsson, and Still [26], who claimthat agile practices do not provide communicationmechanisms in situations where many teams areinvolved in the same development process. Scrum andKanban are for single teams, and not meant for crossteam communication. As a consequence, according toPikkarainen et al. (ibid), they are not tools forcoordinating multiple teams or projects at the same time.Nyrud and Stray [24] identified 11 coordinationmechanisms in a large-scale agile project and concludedthat ad hoc conversations were the most important.In a large-scale setting, the most common strategyfor coordination across several teams is Scrum ofScrum. Scrum of Scrum is a scheduled meeting at whichone team member acts as “ambassador” to participate ina daily meeting with ambassadors from other teams.However, Scrum of Scrum has been found to beinefficient in larger projects.Because of challenges coordinating work in largescale agile, agile consultants have created severalframeworks for scaling agile, such as the Large-ScaleScrum (LeSS) [16] and Scaled Agile Framework(SAFe) [17]. The LeSS framework offers less structureand gives suggestions, tools, and tips for practices thatcan be used for coordination, such as communities ofpractice and scheduled multi-team meetings. In theLeSS, any team or team member should be able andexpected to reach out to another team if there is an issueto be solved (both through scheduled and unscheduledmeetings). The LeSS can be understood as a bottom-upapproach to coordination and gives the autonomousagile team authority to adjust practices. The SAFe seemsto create a structure with more organizational control,which might leave less flexibility for meetings toemerge and for teams to take the initiative forcoordination.3. Research design and methodThe goal of this research is to understand barriers toteam autonomy in large-scale agile development.Hence, studying how multiple teams collaborate isimportant. We designed a holistic multiple case study[35] of 14 teams in three projects in three companies(Table 1). According to Yin, case studies are thepreferred research strategy when a “question is beingasked about a contemporary set of events over which theinvestigator has little or no control” (ibid, p. 9).We collected data through semi-structuredinterviews and retrospectives in three distinct largescale projects across three case companies. Thecompanies were chosen because they participate in aresearch program on autonomous teams. All teams inthe study were working according to the Kanban methodusing some Scrum ceremonies such as the daily standup meeting. While Scrum divides work into a series offixed-length iterations (sprints; whatever is scheduledfor a sprint is the team’s top priority), Kanban benefitsfrom flexible planning because whatever is on the boardis the top priority. Kanban focused on continuousdelivery with changing priorities. Information about thecompanies, the projects, the teams, and the datacollected is listed in Table 1. Sand and Grass are banks,while Necker is a Software Service Provider. Allinterviewed teams are situated in Norway, and allprojects have been running for more than 18 months.In all companies, we interviewed the managersworking closely with the teams, and all team-membersthat were available in the interview period. In Sand, wechose the team that had been working together thelongest, in Grass both teams that were involved in theproject, in Necker one team that was chosen by themanagement. Interviewees were split into twoPage 6999
ties and those without. Interview guides canbe found in Appendices A and B. The interviews lasted40-50 minutes. Data were collected in two rounds. Aftertranscribing and coding a few of the first interviews, theinterview guide was revised based on the feedback. Asecond round was initiated with a revised interviewguide. This round also included some follow-upinterviews that explored particularly interesting subjectsthat were uncovered during the first round.Retrospectives lasted approximately two hours. Theretrospectives were sessions facilitated by theresearchers where the development teams or theleadership team of the project reflected on theirprocesses: what the teams thought was working well,what wasn’t working well, and what they wanted tochange. Afterward, the teams agreed on what measuresto take. We took notes during the retrospectives.Subsequently, we conducted a thematic analysis ofour data as we wanted as few restrictions as possible forour inductive study [5]. Interviews were passed aroundfor transcription and coding, ensuring that everyresearcher had insight into every interview. Researchersalso participated in collective sessions where theempirical material was discussed. The analysis resultedin two themes (overall direction and externalcoordination) with several subthemes.There is a risk that our findings can be explained byfactors that evaded our attention. One reason is that wedid not conduct a retrospective with all teams, and wedid not interview all team members, and therefore,probably missed some subthemes. However, givingfeedback to the observed teams and discussing ourinterpretation of what was going on with the casecompanies helped with validating our conclusions.4. Challenges of autonomy in large-scaleagileTwo themes emerged from the data analysisdescribing the challenges for autonomous developmentteams in large-scale agile, and how they relate toautonomy: overall direction and external coordination.Overall direction is about creating a shared directionamong team member and how goals for the teams areset, what they entail, and how they are communicated tothe team. External coordination entails how teamscoordinate with their environment. This includes how ateam relates to other teams and components, how a teamdeals with additional tasks (that is, tasks that, forexample, are not in the backlog), how teamscommunicate, and how teams coordinate with anexternal customer.First, we will briefly present a general bewildermentregarding the term autonomy as a backdrop for theempirical analysis. A developer from Neckerexemplified how the meaning of the term is not wellknown nor easy to understand, as he had to google itbefore he came to the interview. However, he was notsure if he captured the essence of it. Several developersfrom Grass stressed the necessary knowledge andresources to implement their activities as the mostimportant features of autonomy. A team lead fromNecker added to this, reflecting on how everyone talksabout autonomy without a common definition of it.Further light was shed on this confusion by a productowner from Sand who questioned whether higher-levelmanagers know what autonomy is really about. At thesame time, the informants spoke warmly of autonomy.A business representative from Grass consideredhimself a supporter of autonomous teams, and a linemanager in the same company talked passionately aboutthis way of working. A developer from Necker agreed,highlighting the freedom in how to develop the solutionand working closely together, as features he appreciates.Furthermore, on the question of whether the informantssee their teams as autonomous, the general answers were“yes” or “almost.” The highlighted differences ininterpretations among the informants, as well as theview on their teams being autonomous to certaindegrees, illustrates the difficulty of autonomy.Table 1: companies, the projects, the teams and the data collectedCompanyIndustryProject DescriptionNo. ofteams5Data CollectionSandFinance/bankingChange program forinternal IT services androutinesGrassFinance/bankingWeb program forexternal end-users28 interviews, including tech leads, developers, a businessrepresentative and a line manager.Retrospectives with two teamsNeckerICTSoftware developmentprogram for a large,public customer76 interviews, including team leads and a project manager.Retrospectives on one team and the project level9 interviews, including product owners, a project manager, abusiness representative and a domain architect.Retrospective on one team and on the project levelPage 7000
4.1. Overall direction4.1.1. The Importance of a Shared DirectionSeveral informants emphasized the importance ofcreating a shared direction for the teams in their largescale project. A business representative from Sandemphasized that finding common ground and clarity ofgoals is necessities when implementing autonomousteams:We as a team needed to try to establish what we hadto accomplish. We had to lay that fundament andmake team members familiar with the goals andvision. ( ) We spent many sessions on finding acommon ground.We found that finding common ground is also aboutunderstanding the tasks the team needs to do and theorder in which the tasks should be developed. A domainarchitect from A explained:Before, we didn’t have a plan ( ) but now we are inthe process of making one ( ) the overall goal isclear, but it doesn’t say anything about the order ofthe tasks. The question is whether we should splitthem, or if we should do them as one.Further, to understand the tasks and goals of theteam, the team has to understand the needs of the enduser of the system under development. However, inlarge-scale agile projects, the team is seldom in directcontact with the end-user like in a single-team project.We found team members discussed end-user needs andproblems without having the same end-user type inmind, which resulted in misunderstandings in the team.One team decided to create a set of personas (descriptionof different end-user types and their behavior), to beable to better discuss and understand customer needs inthe team.Having the right competence, network andexperience in the team is an enabler for a shareddirection. We found that experienced team memberswith an overview of the work can help other teammembers and pull the team in the right direction. A teamlead from Necker stated that a team needs someone whopossesses an overview of the team’s assignment. Adeveloper from Grass argues in line with the team lead,expressing that experienced developers are importantnot only because they understand the direction of thework but also understand how to do the work and whoknows what in the large-scale project.While understanding the direction for the team isimportant, we found that it is likewise important tounderstand the direction of the large-scale project whichthe team is a part of. A project manager from Neckerexplained that if the team is missing the larger picture,it is difficult to relate the team’s tasks to it.We found that within a specific large-scale project,the project consisted of the same roles. However, oftenmembers within the same large-scale project did nothave an identical understanding of the roles. Oneexample was the team lead role in Sand. We found thatteam leads, the manager of the program, and teammembers all had different understanding of the role.Some team leads did not even know they were reallyresponsible for leading and developing the teams. As aconsequence, it became difficult to pull the teams andthe project in the same direction within the large-scaleproject. Some teams had members from differentdepartments, and in one team, the members did notunderstand or accept their leader’s authority. A teamlead from Sand described: “I have no control over whatthey do all the time. It is not like I need that, but I haveno power to get things I see as important through in theteam.”A tech lead from the same company explains howthe tech lead role was formed during the reorganizationof the company. The informant explained how he got adescription of his new role but did not remember all ofthe defined responsibilities. The complexity of the techlead role is supported by a project manager fromNecker, who said that it takes a lot of time to understandthe nature of the tech lead role.4.1.2. Setting and Communicating GoalsAs explained in the previous section—shared goals areimportant for a shared direction. In the investigatedlarge-scale projects, we found that goals are often set bymanagement without involving the teams, the goals areoften equal to deliverables and deadlines, and teammembers are not always sure what the goals are.For several teams, goals appeared to be equivalent todelivery deadlines. A developer from Grass explained:“Lately, our only goal is related to deliveries. It is aboutfinishing something at a given time.”When questioning whether the team has any goalsother than specific deadlines, most answered that, ifadditional goals existed, they were not known to theteam. A developer from Grass explained that thedepartments recently set some new ones, but he was notexactly sure what the new goals were. While the goalswere unclear seen from the team members, the businessand management side working with the team had adifferent opinion. A business representative from Grassexplained how the goals are communicated orally, andthat his impression is that the developers have a holisticpicture of what they are doing. However, heacknowledged that there is no arena for creating a sharedgoal between the teams working on the large-scaleproject. However, this is something they are aiming for,Page 7001
as the goals of the departments involved in the largescale project are not aligned today.One explanation for why goals were perceived asunclear to the teams might be that they were oftenidentified by someone outside the team, not involvingthe team members. A developer from Grass explainedthat goal-setting is done mainly by the businessdepartment in the large-scale project. A tech leadsupported this, adding that, after the goals are set, theyare given to the team. When the team sees such goals asunrealistic, they do not commit to them. One developerfrom Grass explained:I think they [the management] set the deadline withthe intention of giving us something to work towards.And then we just have to see if we reach the deadline,or if we have to postpone the date or reduce thescope of our work. In my opinion, the deadlines donot always make sense.While teams were often not involved in defining thegoals, a line manager from the same company expressedthat taking part in identifying goals is an essentialfeature of autonomous teams. While involvement wasdesired, team members acknowledged that they couldnot be a part of all goal-related processes in the largescale project. A tech lead from Sand explained: “Thereare things you have no influence over as a team, becausethey happen on a higher level in the organization, orduring a release process in which other teams areinvolved.”4.2. External Coordination4.2.1. Organizational DependenciesIn a large-scale setting, the teams are dependent on otherteams, projects, departments, and/or systems within theorganization, and vice versa. This is exemplified by adomain architect from Sand, “We do not live in our ownworld (.) one has to coordinate with other teams whoshare components with your team.”The domain architect explained that co-locating withteams who share the same system components is helpfulwhen dealing with this matter. The challenge ofdepending on others is also present at Grass. Adeveloper stated that the team frequently needs to clarifydifferent issues with the business department.Discussing unclear specifications and the need forconfirming decisions are examples of when a teamneeds to make contact before moving on. Further, a techlead from the same company stated that since theworkflow in the project was not synchronized, his teamneeded to clarify issues frequently with otherdepartments. Problems with the synchronization alsoresulted in teams needing to wait for other resources andother teams to finish their part of the job. A team leadfrom Necker explained how they were unable to moveon even if they had finished their own work.We have external dependencies. We had some caseswhen integrating with systems made by externalteams, and they were either not ready or it was notdocumented well enough, or we could not evenaccess it (.) And you always have a lot of casesgoing on that you cannot finish because you have towait for others.Because of the dependencies, the teams were alsoapproached by others. When describing how thebusiness department at Grass communicates directlywith individuals in the team, a tech lead said, “They talkdirectly to those having the task at hand. Sometimes thisis fine, but ideally, they should involve the whole team,so that everyone knows what is going on.”Depending on others to do part of the job reducedthe team autonomy because decision making is limited,the team cannot fully control how tasks are conducted,and they need to adjust their processes to other teamsand actors.4.2.2. Dealing with Additional TasksSeveral informants state that additional tasks (tasksnot prioritized in the spring backlog or prioritized by theteam) delay the teams in doing their initial work. One ofthe interviewees (head of development) from Grass saidthat the team receives a stream of such additional tasks.A team lead in Necker stated that these unrelated tasksmight even postpone entire sprints (the team was usingScrum). The team lead. Therefore. paid attention toexternal actors trying to get the team to do suchadditional tasks. A domain architect from Sand statedthat even though tasks are seemingly unrelated, one canargue that they align with the goals of the team, as thegoals are often very general. A developer in the samecompany stated that such tasks can be, for example,errors in previously developed products that need to befixed right away. Another reason for such tasksemerging is the scale of the development effort. Becauseof the size and complexity of the large-scale project, itseems impossible to plan everything that needs to bedone. Because of all the dependencies, sometimes ateam needs to stop what it is doing and solve new tasksfor other teams before being able to move on.To reduce the challenge of having to deal with toomany additional tasks, most of the teams have one teammember responsible for communicating with the rest ofthe project and organization. In most teams, we found itto be the team lead or the tech lead. According to severalinformants across the companies, there was oneparticular reason why this was often the team or the techPage 7002
lead’s responsibility: Individuals may find it difficult toknow what to prioritize, and when they are approached,they are interrupted. A tech lead at Grass described hisrole as a link between the team and the rest of theorganization. If anyone wants to talk to the team, theyoften approach him. A developer from Grass stated thatbeing interrupted when writing code makes his tasksmuch more time consuming because his work requiresdeep concentration.Several team and tech leads stated that they try theirbest to shield their team from externalities, filtering outwhat they consider as unnecessary for
scale agile, agile consultants have created several frameworks for scaling agile, such as the Large-Scale Scrum (LeSS) [16] and Scaled Agile Framework (SAFe) [17]. The LeSS framework offers less structure and gives suggestions, tools, and tips for practices that can be used for coordination, such as communities of
