In our experience of attempting to understand an organization's readiness for Agile, we see that many organizations do not fully understand the process involved. This in itself can make implementation unnecessarily more complicated. Agile is not a magic bullet and, in the beginning, it can feel like it is breaking down an established process and making things worse.
Having worked with many managers, we have found that a common pattern is not to work with their teams on adoption but to spend their time producing reports and explaining the stumbling blocks, while their teams scramble from one firefight to another. In some of the worst cases, the same people only work on damage limitation, expecting the process to take care of itself. After all, it is only a few documents and a couple of ceremonies, right?
This is symptomatic of organizations that try Agile without a commitment or the knowledge of how to do it right. At the heart of this, it is important to review the conventional command-and-control approach, technical and process shortcomings, and the human dynamic prior to attempting the change. Furthermore, it is crucial to have up-front conversations with all the layers of management to manage expectations of what is about to happen.
How ready are you for the change?
Many organizations feel that by bringing in a ScrumMaster and a coach, Agile adoption starts. This viewpoint is unlikely to lead to success. The fundamental responsibility of a ScrumMaster is to maintain and improve the process and remove blockers on the team level. This in itself is a full-time responsibility. It is not as simple as saying, "We are Agile now, so let's get to market faster at less cost and with higher quality!" Adoption is very much about a change in mindsets, and we meet a lot of people who have had little experience outside of command-and-control environments and find it very difficult to adapt. Rather than forcing Agile on people who will struggle with the approach, try to build team(s) of individuals who will actively support the process.
Conventionally, a project manager will assume responsibility to plan a project, timelines, and costs. The ownership and responsibility of this process sits with a single individual and often does not involve the full team who will be delivering the product. This is where the concept of "being precisely wrong" comes in. In our experience, we have not met a project manager who is an expert in management, in all of the technology and organizational change combined to predict the future with any degree of accuracy, nor in the stumbling blocks they may encounter during the delivery process.
When a "rainbow" team is involved in shaping the plan, you harness the experience of a multidisciplined group who can take "precisely wrong" to "relatively right." When teams feel that they have shaped and own the deliverables, and have anticipated risks and challenges, the outcomes are usually significantly more accurate than in dictatorial environments where team members produce just enough to appease the controller.
Agile adoption involves not just new procedures but also new values. A desire to improve team structure and processes is critical. The project manager and/or ScrumMaster must ensure that, at the end of sprints, retrospectives happen and the team translates its conclusions into actual improvements. Once again, the ScrumMaster/project-manager-as-coach model depends on the project manager's or ScrumMaster's ability to collect good information, analyze it, and make suggestions on organizational and team level processes. When the team is mature enough in their understanding of the process, retrospectives should be left to the team to manage.
By answering a few simple questions, you can start to assess the readiness of your organization for Agile:
Does the organization understand enough about why it needs to adopt Agile?
If the response is, "We want to get to market quicker with higher quality and lower cost," then the answer is no. There need to be quantifiable reasons and a clear, shared vision before proceeding. One of the areas you may want to address is the organization's ability to break down heavy processes and deal with change at a reasonable pace to stay on par with or outpace the competition.
Does the rest of the organization know enough about Agile to be willing to accept the change?
This can be easily done by organizing small-scale seminars, meetings, and/or workshops to introduce Agile to the rest of the organization and thus address any issues or concerns people may have with regard to introducing a new process.
Is there willingness to change?
Introducing Agile to the rest of the organization does not necessarily mean that they are willing to accept it. Assessing the readiness of the organization for this change is a crucial process and can easily be achieved via an organization-wide survey or questionnaire, which should not require any additional time or resources on the part of the organization. What this does is guarantee a basic understanding of whether the organization is ready for a transition or not, and of what needs to be addressed first if a successful transition is to be achieved.
Is this transition supported by the whole or most of the organization?]
Why is this important? This is important because if the transition is being supported by only part of the organization, there is no guarantee of its success. It is important to establish whether this new process is supported by most of, if not the entire, organization and then to identify which subgroups are not viewing it favorably. Address their concerns and help them identify how Agile can best serve their needs.
Is the technical infrastructure and are the teams ready for rapid delivery?
Just a few considerations here: Does your IT infrastructure have the ability to quickly spin up the additional environments required for testing? Do you have a branching strategy? Do you have DevOps implemented, and can you rapidly release when required? Modern development techniques like BDD, continuous delivery, and XP principles all lend themselves to a positive answer. However, the up-front cost of implementing these deters many organizations from putting them in place.
Do you have an adoption plan?
There are a few approaches here, with the most popular being either an organization-wide adoption or starting small and then evolving. For many reasons, a key one being money, very few organizations are comfortable with the first approach. Organizations, more often than not, have to answer to shareholders who are interested in the bottom line, and we have witnessed a number of attempts to move to Agile fail at the AGM (annual general meeting) level.
Having said that, there are organizations that have managed to do this, Salesforce and Atlassian being a couple of examples. Many newly formed organizations follow Agile and Lean practices from the start and are therefore significantly more competitive than linear-delivery companies, by being more disruptive and able to deliver change fast. The key to Salesforce's success was organization-wide support, from CEO to junior members of the team; an effective adoption plan; and first-class training supported by experienced Agile practitioners through regular interactions with the teams both directly and via regular workshops, where challenges were examined on case-by-case basis.
Most organizations opt for the second option: Start small then evolve — and often with IT. This is not a bad place to start, though it's full of stumbling blocks. It is a hybrid approach and, as we know, hybrid methods are usually more difficult to implement, as they require the embedding of a new and innovative approach within an already existing and more archaic one. When people are having to shift between Agile and Waterfall projects, this requires them to understand and incorporate transitions between the approaches that will inevitably impact productivity. Some will naturally prefer one of the two environments — either the structure and predictability of Waterfall or the self-managed (relative) freedom of Agile. That will lead to lower engagement and motivation when they are asked to work in areas they are less comfortable with, which will drag down overall performance.
There are some key questions to ask here.
Is the rest of the organization ready for rapid development, adopting and adjusting to frequent releases and change?
In our experience the answer is usually no. We have to ensure that people working in a hybrid environment understand not just the mechanics of Agile and Waterfall — what makes them different in how they operate — but also why
we might choose one approach over another in different circumstances. We also need to provide an explanation of why one or the other approach was selected for the specific initiative each individual is currently working on; and, when the project uses a combination of hybrid and Waterfall approaches, why the combination was chosen. We also have to provide team members on hybrid projects an understanding of how Agile and Waterfall elements are going to interact — we can't just set up two separate areas of work and ignore the need for work to be passed between those areas. What often happens in hybrid environments is that technologists are in a position to release frequently, but the rest of the organization is not able to absorb or realize the business benefits early enough. This means that a significant up-front cost of transition to Agile by IT does not deliver any quantifiable value for months until the rest of the enterprise is ready to absorb the changes, thus significantly minimizing the benefits of the whole transition exercise.
Are individuals within the organization flexible enough to accept the changes and transitions involved in running an Agile model?
This may seem benign but it is, in actual fact, one of the biggest stumbling blocks that we have encountered in our experience. Organizations often struggle to adapt to what is perceived as a less structured approach. However, removing some of the structure and rigor around project execution (Waterfall in particular) and allowing teams to operate with a degree of autonomy is one of the key principles behind a successful transition.
Does the organization understand the science behind new roles like ScrumMaster or product owner, and what a project manager now does? Is there a need for a conventional PMO?
Again, without a proper introduction to the new processes, the answer is often based on limited perception rather than solid fact — with ScrumMaster being one of the most misunderstood roles in the organizations new to Agile.
One of the other dangers here is to work with people with limited knowledge of Agile and enough seniority to affect the process. Recently we worked with a head of IT who claimed to be an Agile expert. He insisted that each sprint for a newly created team had failed because the team did not deliver everything they committed to. It took some time to turn him around to see that a team needs to find its momentum, also referred to as its "heartbeat," to estimate what can be delivered with any accuracy.
Implementing Agile within an already existing structure is not an easy process. In fact, it is quite a challenging one, as it takes place at two levels concurrently: one being the project level, where development teams are taught to build software using Scrum; and the other being the organizational level, where change is being implemented. One cannot work without the other; the first improves software delivery, and the second remedies impediments to productivity identified in the first. Without removing these impediments, the whole process may not be successful. As such, it is really important to consider the following steps:
- Assessing the readiness of your organization for Agile. This could be achieved using Scrum presentations with a view to present concepts to the entire organization and raise general awareness, a first crucial step toward assessing "organizational readiness" and defining the next steps. Once this is achieved, you can define plans, identify potential pilots, schedule training, and resource the pilot project(s).
- Providing initial training for the early participants. The second important step is to provide training to the various participants; i.e., training the ScrumMasters to run the pilots and the product owners to maximize ROI using Scrum. Then review and modify metrics that monitor the use of Scrum within the organization and define the value derived from the pilots.
- Identifying the impediments. The last but not least important step is to identify impediments and establish a backlog for tracking and evaluating impediments that arise during the pilot projects. This backlog will be your support mechanism for change within the organization and will help you have a better perception of the actions required to implement change.
An ideal place to start would be to use Agile in one or more actual projects. Closely watched and monitored, these pilot projects will help identify organizational obstacles that can prevent effective Agile practices. Once they have been identified, they should be fixed on the spot (where possible) and recorded in an Organizational Change Backlog for later attention. Before proceeding to scale Agile to larger project environments, an organization needs to look at these impediment types, categorize them by area, and identify various scenarios to mitigate or remove them.
A common recommendation is that pilot projects should be of a medium duration and importance, typically three to six months. The main reason for this is not to make the mistake of putting a mission-critical project through Agile at this stage. Pilot projects still need to be delivered, but they are also there as a mechanism for identifying detail behind the organizational and procedural stumbling blocks. The ability of your organization to adapt and transition through the phases and methods, or any resistance to the whole process, can provide a clear indicator as to its readiness for the new model.
Agile in itself will not give you full benefits without being supported by appropriate tool sets and modern technical approaches. Developers who do not develop unit tests and write code using TDD/BDD style, refactoring older codebases using modern techniques, lose out on significant benefits of modern development practices. QA engineers who do not use automation will spend too much of their time on regression instead of feature, exploratory, and destructive testing. Inevitably, this will prevent QA from doing their work in the sprint and they will fall behind development. This often leads to a shift back to silos, with QA working separately from Dev, and the "us and them" attitude creeps back in. Ideally, the right practices need to be backed by continuous delivery to allow technology to concentrate on solving challenges instead of administration. Below is a useful diagram explaining the principles behind continuous delivery:
Teams should look to Agile application lifecycle management (ALM) tools that manage backlogs, support planning, and enable reporting to support their Agile approach. There are quite a few to choose from: Jira/Confluence, TFS, and Swift ALM, among others.
Do you understand what information radiators are required?
Communicating the state of Scrum projects can be achieved using simple and powerful information radiators, like whiteboards for colocated teams, task boards, product and release backlogs and the project, and program burn-down or burn-up charts. Ideally, all this would be in one place, with easy access for information consumers. Distributed teams can use online equivalents. Although this is not a replacement for face-to-face interaction, what has worked well for us is to use a tool like GoTo to create an interactive environment when teams cannot be in the same location, and run a permanent chat window where the whole team can discuss daily matters.
Many teams struggle to work out what information radiators are needed during the adoption process. The answer is as many or as few as will satisfy the needs of the organization. More often than not, organizations demand a lot of feedback at the start, gradually reducing it to what is actually needed as the time moves on and people start to understand the changes taking place. Take time to scope what your observers need and set up relevant dashboards.
The start of adoption is usually a little on the chaotic side. A word of caution: Try to strike a balance between the good news that the process is bringing and the challenges you are discovering on your journey.
The value of communicating customer needs
In an ever-changing world where we are all in global competition mode and innovation comes from all angles, staying on top of everything can be tricky. What never ceases to surprise us, beyond the obvious, is how few teams understand what a value proposition is and what customer benefit they are delivering as a part of their work. Senior management are usually more in tune with the vision and value proposition, but this information seldom travels far enough down the ranks for the teams to fully understand the solutions they need to be developing. It is very uncommon for a conversation between the business units and IT to start with: "Our customers are asking for the following and this is why. How can IT solve this?"
Instead, conversations normally focus on what needs to be done, without necessarily enough background. Imagine trying to solve a challenge without knowing what the precise challenge is or being consulted on the desired outcome. Agile lends itself very well to rapid prototyping, and when teams are fed the right information and feel empowered to contribute, they will not just produce solutions fit for purpose but potentially offer a great deal more. What is often missed during these interactions is an understanding that technology knows the business and technology combined, as well as the latest techniques for achieving the desired outcome. The thought process needs to shift from the conventional servant (IT) and a leader (business) models to a partnership, one where the business concentrates on "what" needs to be done and technology looks at "how" to achieve it and, together, the conclusions are reached.
Customers often pressure development organizations to deliver features faster than is feasible. Some organizations accommodate this by compromising on the quality of the product, omitting refactoring, cutting QA efforts and other solid engineering practices. Organizations that succumb to this pressure eventually build systems that cannot be effectively maintained or enhanced, leading to a substantial cost of rewrite and re-release of the software. To avoid this, customers can be invited to participate in the implementation process and to witness for themselves the value of good engineering practices. This will further reinforce positive customer engagement practices and enhance the organizational and cultural benefits of Agile.
Diving into Agile headfirst, without proper consideration for the cost and the effort involved or without realizing why you want to implement this method and the challenges involved, will result in a costly and unfulfilling experience for the teams and the organization. It is important to invest enough time up front to understand the challenges and how Agile can solve them, prior to starting the whole process. Our advice is to conduct a robust feasibility study to assess the needs of your organization and answer such questions as what it is that you think Agile would do for you and the expected outcomes of the process. This feasibility study should come as part of your package when hiring an Agile professional and is one of the founding blocks for a successful and well-managed Agile transformation. This is also the best way to ensure that your organization is not wasting valuable resources (time, money, and workforce) in implementing a one-size-fits-all solution that may or may not work for you; it will instead help ensure that the solution you are being offered is tailored to your organizational needs.
The author gratefully acknowledges Lilia Stobbs for her great brain, Endless Patience, and first-class editing.