Agile has grown in popularity as a credible alternative to traditional software development processes. More and more companies are adopting Agile to realize the benefits of quicker delivery and better-quality products that reflect user needs. However, other companies become disenchanted with Agile practices. Let's analyze some of the key reasons why some companies fail in adopting Agile practices -- and what we can do to fix that.
Agile is not seen as a change in mind-set
Many companies think that the transition to Agile practices is a linear process and that there is one-to-one mapping between Agile models and other traditional models, like Waterfall. So they look for a journey from Waterfall to Agile and often ask Agile coaches to tactically guide them through the conversion process. The Agile coach is looked upon as a process expert expected to outline a step-by-step path that will result in a clean transition from a traditional process to Agile. However, these companies fail to realize that adopting Agile practices is more of a culture and mind-set change than a process change. Plus, there is no one-to-one mapping between the two processes.
Key to the success of Agile practices is the adoption of its core values. The first thing any company or team should do before trying Agile is to set aside traditional models and focus on core Agile values, such as communication, collaboration, feedback, incremental delivery. They should get out of the mind-set of following the big-bang approach and start thinking "lean," in terms of specifying value in the eyes of the customer, eliminating waste, empowering employees by creating self-organizing teams, and continuously improving in pursuit of perfection -- with no endgame. For example, if a company gets 100 user stories from a customer, then it is really not important to work on all of those 100 stories; better to collaborate with the customer to deliver the top 7 to 8 percent of those stories first, and then continue to progress incrementally. Work with the customer to deliver maximum value with the minimum possible features instead of fulfilling the negotiated contract. Many studies indicate that the top 7 to 8 percent of features provide the maximum bang for the buck.
It is fallacious to think of adopting Agile without adopting its core values.
Fear of failure
In an organization, any kind of change meets with resistance. It is really not hard to imbibe the Agile thought process, but most companies have no such experience and skill, so they fear this change. They have a financial goal to meet and they fear losing illusory control in order to meet that goal, which results in hybrid Agile adoption. Teams try to retrofit Agile into their existing processes to become fail-safe -- but in reality that itself becomes the reason for failure, because they never learn the real value of Agile adoption.
So how should a company (or team) overcome this fear of failure or the fear of the unknown? The best way is to embark on the Agile journey bravely, with the most complex and critical thing first. I don't recommend going with the low-hanging fruit first, because that reflects the lurking fear in the mind. Companies/teams should shed the illusory control and assimilate the values of Agile practices and work hard to be successful with Agile. Agile is simple, proven, and time-tested, and, more important, it works. Remember that changing mind-set and adopting a new culture take time, but once you start seeing small wins you should start replicating the success with other teams. At a later stage I would recommend hiring an external consultant to further solidify the gains of Agile adoption.
Adopting Agile without working hard to make it successful
Companies often get entrapped in mystical thinking -- Agile is the panacea for all organizational ills, Agile adoption will magically remove all dysfunctions and start delivering quality software swiftly. They make the decision to Adopt agile practices with high hopes. But soon reality strikes, and they realize that development has actually slowed down, or that it has improved somewhat but not to the degree they expected. Disillusionment sets in and they start wondering if they made the right decision in adopting Agile. The truth is that Agile never fails (if implemented correctly); it simply brings up the dysfunctions of the organization. Some of the key symptoms to watch are:
Constantly failing iterations
Teams not honoring commitment
Lack of cross-functional teams
Increasing reliance on an Agile coach
Inability of teams to continually improve when things are going deceptively well
Prevalence of detailed documentation over interactions
Companies that latch onto those warning signs and make an active effort to eliminate those dysfunctions often succeed with Agile adoption. Others, lacking the will to remove those dysfunctions, try to make adjustments in the process and drift away from realizing the true potential of Agile adoption.
Traditional organizational structure with the new rules of engagement
Adopting core Agile practices require agility in organizational structure as well. A lot of traditional roles, like resource managers and project managers, find themselves not so well aligned with the new rules of engagement. On the other hand, key Scrum roles such as the product owner, ScrumMaster, and the Scrum team may not exactly map with the traditional roles of the organization. While adopting Agile, organization structure should be tweaked to meet the needs of Agile, not the other way round.
The Agile coach should not be seen as a leader, making decisions for the team. All decisions should be taken by the self-organizing team, and the Agile coach should only help with impediments and the overall process, acting as a guide for the team to get them through difficult terrain in uncertain times.
There is also a common perception that only development teams need to be trained in Agile practices, and that other line managers should continue to function the way they already do under a traditional process. This approach is counterproductive and creates a wedge between the traditional managers, oblivious to Agile/Lean values, and the relatively more Agile development teams, who keep looking for inspiration from the parallel universe. So it is super-critical that all members of the team (including managers) be trained in Agile practices to create a value chain in the organization.
Being too liberal or too strict with Agile practices
Some companies/teams follow Agile principles so strictly that they ignore the true intent behind the practice. Teams become so "fundamentalist" about Agile practices that they kill creativity and create a perception that process is more important than the product. On the other end of the spectrum, some teams customize Agile models so much that what they have has completely lost any Agile meaning. For example, the daily stand-up meeting becomes a status meeting instead of a replanning meeting; the retrospective meeting becomes another ceremony without any conscious effort to learn from the previous iteration or remove dysfunctions; the sprint planning meeting turns into a story distribution meeting, where developers carry orders instead of being creative problem solvers; and then there's the introduction of mini-Waterfall within an iteration. Soon companies come to believe that Agile practices are no better than or different from any other prevalent practices.
This is not a complete list of dangers, for sure; but companies/teams will gain significantly if they focus on these key areas and work hard to understand the true intent of Agile practices before embarking on the Agile journey. Adoption of Agile without believing in its core values is an experiment in futility.