Agile Adoption

How to Successfully Transition an Organization to the Scrum Framework

17 March 2014

Robert Weidner
Marshfield Clinic


The change effort required to effectively and efficiently transition an organization to Agile is significant. Following a relevant change model, and iterating through each of the process steps and stages, can ensure a successful implementation. The long-term effort can then be sustained through the use of review and adaptive planning. This paper provides an outline for transitioning a medium-to-large-scale organization to Scrum.

Introduction

In a large organization, it can often take up to two years to implement Scrum. This is a big change in culture, and change is hard work. Not only does the organization need to adopt new practices but it needs to value new principles. This change in philosophy will be in stark contrast to the old command-and-control style approach that likely preceded it. Software development organizations are moving away from traditional, Waterfall-based project management methods to Agile frameworks. These organizations are seeking a competitive advantage by removing process-heavy techniques that rely on obstructionist change-management approval processes, and instead they are adopting a more collaborative and adaptable approach.

Making the transition from a traditional Waterfall-based method to the Scrum framework will require a significant change effort. To effectively manage the adoption process, and then sustain the change, it will be beneficial to use a formal change model. The organization should develop a change strategy and then measure their progress against the plan. Like the Agile process being implemented, progress should be monitored against the plan, and the plan should be adapted based on feedback and/or in response to unintended consequences.

The change process

The process of change management includes the following steps (Hayes, 2010):
  1. Recognize need and start change process
  2. Diagnosis
    1. Review the present state
    2. Identify the future state
    3. Quality of the vision
  3. Plan and prepare to change
  4. Implement the change
  5. Sustain the change
In addition, throughout the entirety of the change effort, the following ongoing activities will need to take place:
  • Review progress and feedback
  • Manage the people issues
    • Power, politics, and stakeholder management
    • Leadership
    • Communication
    • Motivating others to change
    • Support for others to help them manage their personal transitions
    • The change manager's mode of intervening and the effect this has on the change relationship

Change models

Lewin (1951) suggested a three-step model for change that included the following stages:
  1. Unfreezing
  2. Moving
  3. Refreezing
During unfreezing, driving forces should be emphasized and restraining forces should be weakened. This will destabilize the current environment and ready the change. It should prompt people and teams throughout the organization to begin letting go of old behaviors and seek out new and more effective alternatives.

When moving, the driving and restraining forces are adjusted to the point that the organization shifts to a new level. This can be accomplished by modifying attitudes, beliefs, processes, systems, and structures. Smaller changes may only adjust a subset of these attributes, while larger changes will require that all five be modified.

Once the organization has shifted to a new level, refreezing occurs. This allows the change to be reviewed, progress measured, and the ongoing effort fine-tuned. In addition, the change is reinforced, and regression can be avoided. Organizations often perform the first two steps in Lewin's change model but, if they exist in a turbulent environment, will forgo the third step. This is a dangerous scenario, because refreezing allows the change effort to be sustained. Without this crucial step, employee behaviors may revert back to old roles and old habits.

Like the Agile process itself, which is based on the kaizen principle of "small changes," Lewin's three-step model uses an iterative and incremental approach to transition and sustain the effort. The organization should make small, progressive changes and then freeze the change effort to allow for adaptation based on review, and to embed the new practices. This method is consistent with the gradualist paradigm, which posits that "fundamental change (organizational transformation) can occur through a process of continual adjustment, and does not require some major discontinuous jolt to the system in order to trigger a short episode of revolutionary change. According to the gradualist paradigm, change is evolving and cumulative" (Hayes, 2010, p. 38).

However, the punctuated equilibrium paradigm (Hayes, 2010, p. 24) acknowledges that incremental changes occur when the focus is on continual process improvement, during periods where the industry is in equilibrium. In this scenario, small adaptations are implemented to gain a competitive advantage. Meanwhile, a fundamental change at the deepest levels of the organization cannot be sustained through incremental efforts. Incremental changes should be used for optimization, while transformational (i.e., revolutionary) change will require breaking with the past and forming new patterns. The focus in transformational change is to do things differently, rather than doing them better. Sometimes it may even include the decision to do different things. This is because transformational changes occur during periods of disequilibrium and may call into question the very purpose of the organization.

Once implemented, the Scrum framework is well suited to the gradualist paradigm, since the focus is on continual process improvement. However, when an organization considers switching to the Scrum framework, this is often during a period of disequilibrium and thereby will initially require transformational change, which should then be followed up with small, incremental optimization efforts.

Meanwhile, open systems theory focuses on the interrelated components between the organization and the larger system it is embedded in (i.e., environmental conditions) that are causing the need for adaptation. It is a quest for internal and external alignment, with no specific end state but rather a focus on continual optimization. Scrum is closely aligned with open systems theory, since they share the following characteristics (Hayes, 2010, pp. 93-94):
  1. Embedded within a larger system
  2. Able to avoid entropy
  3. Regulated by feedback
  4. Subject to equifinality
  5. Cyclical in their mode of functioning
  6. Equilibrium seeking
  7. Bounded
The Scrum framework is embedded in a larger system that requires customer engagement, alignment with organizational directives, implementation of regulatory requirements, and adaptation to emerging industry trends. It is able to avoid entropy through adaptive planning techniques and customer collaboration. Scrum uses product and process feedback loops (e.g., sprint review and sprint retrospective ceremonies) to review outputs and then regulate inputs for the next iteration. Scrum is subject to equifinality, since it emphasizes continual process improvement and thereby adapts process techniques to the unique circumstances of each team. Scrum uses sprints, otherwise known as iterations, which are cyclical in their mode of functioning and use a repetitive cycle of input, throughput, and output. The Scrum framework seeks equilibrium through patterned events and sustained pace. Last of all, Scrum is bounded by limiting the amount of work in progress (WIP) at any one time.

Realignment

George Schlitz (2013) described five facets of an organization that may need to be altered when switching to an Agile framework. These include:
  • Execution
    • Skills, practices for individuals and teams
  • Delivery
    • Product development and delivery
  • Product and business model
    • Product management and strategy
  • Organization
    • Structures, processes, and systems by which work gets done and is organized
    • Collective beliefs, perspectives, and habits by which people make sense of things
  • Leadership
    • Leadership and management styles and beliefs about what constitutes effective leadership and management
    • Orienting vision and environmental design
Schlitz recommends that each of these facets be considered and managed throughout the change effort. The current state of the organization should be defined, and the future state identified, with the necessary changes being made across all five facets using a "build-measure-learn" feedback loop, which closely resembles Lewin's "unfreeze-move-refreeze" approach.

According to Mike Cohn (2010), if the implications of becoming Agile are not transferred into other parts of the organization, then organizational gravity will kick in, thereby reverting the change effort back to the previous state of existence. In his book Succeeding with Agile: Software Development Using Scrum, Cohn states the following:

Organizations and organisms evolve to fit their environments. According to the principle of selection, those traits most likely to help an individual or group survive in the organization will be the ones retained. It is the organization's leaders and managers who define what traits help groups or individuals survive. If agile values such as openness and transparency lead to survival in the form of promotions and public praise, those behaviors will be the ones individuals select.

Cohn describes four components that may require realignment under Agile:
  • Human resources
  • Facilities
  • Marketing
  • Finance
For instance, human resources should consider modifying personnel evaluations so that they emphasize team behaviors rather than individual performance. By shifting to team-oriented criteria, the organizational environment will emphasize the values inherent in Agile frameworks and thereby promote individual adoption of the desired behaviors. Many organizations are uneasy about switching to a group accountability model, since they have an intrinsic preference for the traditional, individual approach (Katzenbach and Smith, 1993, pp. 3-4).

Similarly, frameworks like Scrum use colocation to emphasize collaboration and collapse the feedback loop. Consequently, facilities may need to be remodeled in order to create an open, collaborative environment that will be conducive to this new approach. A "caves and commons" design is recommended (Stenbeck, 2013). In this design the team is colocated in a common, open environment but has access to a shared but private individual work space known as a cave. Stewart Brand advocates an Agile (i.e., iterative and incremental) approach to space design (1994):

Sullivan's form-follows-function misled a century of architects into believing that they could really anticipate function. Churchill's ringing and-then-they-shape-us truncated the fuller cycle of reality. First we shape our buildings, then they shape us, then we shape them again -- ad infinitum. Function reforms form, perpetually.

This sentiment echoes the architectural perspective of Christopher Alexander (1964):

Things that are good have a certain kind of structure. You can't get that structure except dynamically. Period. In nature you've got continuous very-small-feedback-loop adaptation going on, which is why things get to be harmonious. That's why they have the qualities that we value.

Thus, the organizational space design should grow and adapt to conform to the changing dynamics of the team(s). Space equals culture. If the organization values the Agile principles of transparency, collaboration, responsiveness, adaptation, and collective ownership, then an environment design conducive to those principles is essential. The space design should augment the team and reinforce the values and behaviors of the change effort.

Adopting Agile principles requires a shift in focus for Finance to an adaptable model that welcomes changing requirements. The emphasis is placed on customer collaboration rather than contract negotiation. Consequently, well-defined contracts that prohibit change requests should be limited in use, largely replaced by generalized verbiage that allows the two parties to negotiate throughout the development effort. This is often accomplished through timeboxing the activity while leaving scope flexible, and then collaborating with the customer to ensure that the most important items are being completed first.

Making the transition

Unlike other change efforts, Scrum adoption can be particularly challenging because it requires implementation from the top down and bottom up, simultaneously. Scrum is about team empowerment, so top-down enforcement is not an ideal scenario. Instead, leadership needs to demonstrate an organizational commitment to Scrum and prompt the change effort, but afterward they have to empower the teams and let the change emerge throughout the organization.

According to the whitepaper A CIO's Playbook for Adopting the Scrum Method of Achieving Software Agility (Schwaber, Leffingwell, and Smits, 2005):

The speed of Scrum implementation is directly related to the:
  • Degree of change required within the organization
  • Urgency within the organization to improve its software development and delivery process
  • Effectiveness of leadership within the organization
In alignment with the steps of the change process outlined by Hayes, the first step should be well underway at this point: The organization has recognized the need (or desire) to switch to the Scrum framework and is initiating the change process. This should prompt the next step, diagnosis, during which the present state is reviewed, a desired future state is defined, and the quality of the vision is clarified.

During the diagnosis phase, tools and assessments should be used to gain organizational insight. Both SWOT and Force Field Analysis are good tools to use early on, because they are quick, collaborative, and insightful. Once a foundation has been acquired with these rudimentary tools, then it can be progressively elaborated into more complex models. The Burke-Litwin (1992) causal model of organizational performance is well suited for Scrum, since it is an open systems model that looks at every level of the organization, and the interaction of each element. This way, the business concerns, and subsequent consequences, can be examined to determine the full impact of the change. The diagnosis should provide the organization with a road map for the change effort, showing the juxtaposition of the desired state against the current state. In addition, it should help identify how each element interacts, and what potential consequences a change to one element may have to another interrelated element.

Next, the organization must plan and prepare to change. Using the information obtained during diagnosis, organizational barriers to the change effort should be targeted for removal. This is akin to Lewin's unfreezing stage, during which the drivers are emphasized and the resistors weakened.

In addition, it will be imperative to consider the role of space design in the change effort. Scrum works because it reinforces Agile principles through the alignment of philosophy and practice. Colocation is an essential ingredient for creating a collaborative environment with a reduced feedback loop. If there will be distributed team members, then collaborative tools that will augment team efforts and emulate high-bandwidth communication will be necessary.

Stewart Brand indicates there are six things that should be considered when reshaping work environments, knows as The Six S's (1994):
  • Site (i.e., location)
    • Rate of change: Extremely low
  • Structure (the foundation and load-bearing elements)
    • Rate of change: 30 to 60 years
  • Skin (exterior surfaces)
    • Rate of change: 20 years
  • Services (electrical/communications wiring, plumbing, sprinkler system, HVAC)
    • Rate of change: 7 to 15 years
  • Space plan (interior layout – where walls, ceilings, floors, and doors go)
    • Rate of change: Every 3 years or so
  • Stuff (furniture)
    • Rate of change: daily to monthly
Brand also points out that the oft-quoted cliché "form follows function" overlooks one critical element, and thus should be amended to "form follows funding." Colocation can be a significant change for introverted developers who are used to working in isolation, so the impact of this change on morale should not be underestimated. The great thing about Scrum, and its predominant use of low-tech tools and techniques, is the ease with which it can be implemented, and the subsequent adoption rate throughout the organization. Often, teams will move into a colocated space, setting up workstations on folding tables. This will work for the short term but should not be viewed as a long-term solution. Eventually, funds should be allocated to specifically design the space with Agile principles in mind. Do not leave teams in an inadequate workspace for too long, as this will stifle the change effort, reduce morale, and build resentment.

With the ground readied, the organization should then proceed to implement the change. This will be the moving stage of Lewin's three-step model. The organization should begin with a pilot effort that will combine projects, and team members, that are ideally suited for the change and can then serve as influencers for the overall adoption effort. The pilot team should continue to optimize and adapt course as necessary to improve performance. In Scrum, the team should be self-organizing and, consequently, self-regulating. Retrospective ceremonies should be held at the end of each sprint so the team can reflect on their previous effort and continue to fine-tune process efforts. Scrum uses adaptive planning and empirical process control to manage complex projects and gain a competitive advantage (Schwaber, 2004).

A gradual adoption process is well suited for rolling Scrum out in larger organizations. Initial efforts should be focused toward "assessing the organization's readiness for agility; providing initial training for the early participants; and building the product backlog for the initial projects." A first pilot project should be identified. This would "demonstrate the positive benefits of improved software agility within the organization," thereby increasing interest and excitement for others to try out the new process. After a few successful pilot projects have been completed, it would then be time to "expand the usage of Scrum and its benefits to a significant subset of the development organization" (Schwaber, Leffingwell, and Smits, 2005).

To sustain the change, leadership should also focus on tracking and removing organizational barriers that are a hindrance to team performance. This uses the Scrum process for software development and then applies it to the organizational transition effort. To do this, a list of organizational impediments should be identified and tracked (this can be done through a backlog), with the goal of removing anything that obstructs the process and subsequently removing identified waste. In Lean, this is known as value-stream mapping. Thus organizational change efforts should be refrozen, reviewed, and then retargeted for continual process improvement.

As pilot projects are met with success, the organization should begin to expand its efforts. A seed approach is helpful here (Cohn, 2010), where a member from a previous Scrum team will join a new team and provide them with process-knowledge transfer. The organization should continue to loop through Lewin's three-step model of unfreezing, moving, and refreezing, all while expanding their efforts and adapting to unforeseen consequences.

Managing the people issues

When confronted with "too much change in too short a period of time," people may become disoriented and experience future shock (Toffler, 1970). According to Mike Cohn, many organizations and employees "have been suffering from future shock for years" (2010). He cites the following causes:

Teams are asked to do more with fewer people. Outsourcing and distributed teams have become increasingly common. These adjustments were preceded by the rush to move applications to a client/server model, then onto the web, and then into services. Add to these the constant, and constantly accelerating, rate of change in technology itself -- new languages, new tools, new platforms -- and future shock is now. It should not be surprising that transitioning to Scrum can often be the change that pushes people into future shock. The pervasive nature of adopting Scrum and the fundamental changes it causes in how people work and interact have a higher risk of triggering the future shock effect.

An external change agent, such as a Scrum coach, could be brought into the organization to provide an objective point of view. They would act in a consulting role, offering organizational advice and facilitating the process. However, the change initiative would have to belong to the business, and it would have to be aligned with the organization's overall strategic vision, culture, and desire. The change must derive from within (i.e., internally, rather than externally). Consequently, leadership must become the champions of this effort and advocate on behalf of it. While some teams will be excited to transition, and may event start using Scrum practices on their own, for the entire organization to transition and then sustain the change, the effort must be reinforced by leadership.

The CIO must serve in the role of ScrumMaster for organizational change. According to Ken Schwaber (2005), the differentiator between success and failure for top-down implementations of Scrum is the CIO. Therefore, it will be imperative for the CIO to be heavily involved in the transition effort: advocating the process, removing organizational impediments, and acting as a change agent (rather than just a change catalyst -- i.e., someone who initiates the change but then no longer takes an active role to ensure its long-term success).

Leonard Sayles points out the contradictory nature of the leadership role in an open systems model (1964):

The one enduring objective is the effort to build and maintain a predictable, reciprocating system of relationships, the behavioral patterns of which stay within reasonable physical limits. But this is seeking a moving equilibrium, since the parameters of the system (the division of labor and the controls) are evolving and changing. Thus, the manager endeavors to introduce regularity in a world that will never allow him to achieve the ideal.

Sayles has correctly identified that environmental conditions (e.g., market demands, regulatory requirements, pace of technological innovation, etc.) continue to evolve, and thus the focus should be on continual process improvement rather than toward achieving a singular, static, and complacent state of existence. This is precisely why the servant leader approach of Scrum is so effective . . . because the leader must serve the team, rather than vice versa. The team, and thereby the organization, should be in a state of perpetual adaptation.

In what is equally good advice for both people and organizations, the following quote from Erich Fromm is an excellent reminder about the need to grow and adapt in order to remain viable:
 
We must be in order to become, and we must become in order to be.

References

Alexander C. Notes on the Synthesis of Form. London: Oxford University Press. 1964.

Brand S. How Buildings Learn: What Happens after They're Built. United States: Viking Penguin. 1994.

Burke W, Litwin G. A causal model of organizational performance and change. Journal of Management. 1992.

Cohn M. Succeeding with Agile: Software Development Using Scrum. Boston, MA: Addison-Wesley Signature Series. 2010.

Fromm E. To Have or to Be? New York: Continuum. 1976.

Griffin R, Moorhead G. Organizational Behavior (10th ed.). Canada: South-Western, Cengage Learning. 2012.

Hayes J. The Theory and Practice of Change Management (3rd ed.). New York, NY: Macmillan. 2010.

Katzenback J and Smith D. The Wisdom of Teams. United States: HarperBusiness. 1993.

Lewin K. New frontiers in group dynamics, in D. Cartwright (ed.) (1952) Field Theory in social science. London: Social Science Paperbacks. 1947.

Sayles L. Managerial Behavior: Administration in Complex Organizations. New York: McGraw-Hill. 1964.

Schlitz G. Five facets of the Agile organization: Holistic change for the serious. 2013. Retrieved March 2, 2014, from http://www.bigvisible.com/2013/06/five-facets-of-the-agile-organization-holistic-change-for-the-serious/.

Schwaber K, Leffingwell D, Smits H. A CIO's Playbook for Adopting the Scrum Method of Achieving Software Agility. United States: Rally Software Development Corp., and Scrum Alliance. 2005.

Schwaber K. Agile Project Management with Scrum. Redmond, WA: Microsoft Press. 2004.

Stenbeck J. PMI-ACP and Certified Scrum Professional Exam Prep and Desk Reference. La Mesa, CA: GR8PM. 2013.

Toffler A. Future Shock. United States: Random House. 1970.




Opinions represent those of the author and not of Scrum Alliance. The sharing of member-contributed content on this site does not imply endorsement of specific Scrum methods or practices beyond those taught by Scrum Alliance Certified Trainers and Coaches.



Article Rating

Current rating: 4.9 (7 ratings)

Comments

Be the first to add a comment...


You must Login or Signup to comment.