After years of helping organizations large and small to transition to become more Agile, I have learned one thing: The larger the organization, the more Lean thinking helps to be effective while using Scrum. Lean is not as necessary for smaller organizations with independent teams; however, it is vital for larger development organizations, say of 100 or more. If you work in a larger organization, this article is for you.
Forget what you have heard about Lean being derived from Toyota and manufacturing. While true, it gets in the way of understanding. Rather, think of Lean as a body of knowledge, mindsets, approaches and tools that apply to the work we do (1). Lean is composed of three main bodies of knowledge: Lean science, Lean management and Lean learning. In this article, I will describe each of these and include a description of how they help Scrum teams.
Wikipedia (Wikipedia, 2010) defines science as:
"an enterprise that builds and organizes knowledge in the form of testable explanations and predictions about the natural world"(2).
Software development has a science with, admittedly, considerable unpredictability in it. However, even though it is not fully predictable we can still apply scientific processes to it (3). I will use the term “Lean Science,” to mean the knowledge of how software development and product development works. Agile already describes some of these “rules” or “laws.” One “law” of software development is that, for large projects, you cannot figure out everything up front. It is not a lack of ability that makes you unable to figure everything out up front; it is just the way it is. This leads to approaches such as “build things in stages and see how to respond with what we’ve learned” – what Agile/Scrum calls iterative development with “inspect and adapt” as a learning model.
There are quite a few of these laws. Some of the most important ones that relate to software include:
- All things being equal, the more time that elapses between writing code and testing it, the longer it will take to fix it if an error is discovered.
- Communication between people of different backgrounds (e.g. customer and developer) about things that do not yet exist (e.g. software that is to be built) will not be completely understood by either party and some of these misunderstandings will not be noticed.
- Working on too many things at once makes productivity go down and defect rates go up.
- Knowledge tends to degrade over time. This implies that defining requirements too early will lead to having to redo them or forcing you to work from old information.
- Shortening cycle-time (the time from starting work on the concept until it is consumed by the customer) will lead to improvements in quality and productivity while lowering cost (4).
Experienced Agile developers know many other such laws based on their past experience. And these laws, being true, require us to modify our practices to align with them. In many ways, Scrum is designed to deal with these laws.
For example, the practice of iterative development is itself a method of not doing too much analysis up front. Getting feedback from the customer at the end of every sprint is an attempt at discovering our misunderstandings quickly. A two-week time-box for a sprint naturally limits how many things can be worked on. The team-swarm (two or more team members working on the same story) also limits the amount of work being done at one time. And one of the more important Scrum practices – having a team focused on one product backlog – is following a fundamental law – co-location and focusing on one project increases productivity and quality (5).
Scrum is one practice to achieving short cycle-times by incrementally building software with rapid delivery when possible. Kanban software methods take this a step further. Kanban is another practice, also based on Lean science, that takes into account the human aspect of change – that too much can be daunting and that discarding what people already know can be considered disrespectful. It is designed to improve an organization’s ability to deliver high quality software faster. Kanban allows organizations to start where they are, including its team structures and roles.
To change its methods, the organization first examines how its processes are working and then seeks to improve them by limiting the amount of work-in-progress (WIP) at each step. The underlying tenet is that delays cause defects as well as increasing the time to fix the defects. Kanban creates visibility into the impediments slowing down work and provides a way to improve at a rate the teams find most advantageous. The largest factor in slowing down work is typically working on too many things. Therefore, Kanban suggests setting limits on how much work can be done at each step of the way.
In Kanban, teams develop explicit policies about the work process: about limits on WIP, about priorities of work, about entry and exit criteria, and so on. The team agrees to abide by these explicit policies and to revise them if the policies are helping or hurting the flow of work. One advantage of this is that it allows teams to adjust appropriately to current conditions. Another advantage is that it allows teams to learn together as they discuss the process and their beliefs about the work being done. Explicit policies reflect the current understanding of the team.
This latter point is powerful. When teams talk about their work process in the midst of doing their work, they can make changes while it still matters. This is usually better than waiting until the end of a sprint to hold a retrospection about what has happened after everything is over. Such retrospections often become stale exercises. And discussing work styles and approaches on a daily basis via the explicit policies helps the team gel and keeps the team focused on the affects of changes to work practice. My experience is that having these conversations does not result in extra meetings or time away from work and, in any event, not having agreement on how to work can cause considerable challenges for the team.
Much of current thinking on management appears to come from one of two paradigms. Either management knows best and tells teams what they need to do or teams know best and believe it is sufficient to tell management what they are working on and what their results are. The first approach leads to micro-management and absurd demands for getting more work done. The second leaves management in the dark, unable to assist the teams when they need it. It implies management’s role is only to set up the work for the team, but not to be involved in helping achieve the work. This reflects a common fear that management will interfere with the work.
What I call “Lean Management” offers a third approach: the manager as leader / coach. It starts from the position that management and teams need a common ground for conversation. While team members will better understand how to do their work, management will have a better perspective on the enterprise view and how to best create the context for the teams to work together. This approach marries the local knowledge of the team with the broader perspective of management. Each play a part in the collaboration on how doing the work of the organization can best be accomplished.
This role of leader / coach is critical and is more than simply a facilitator or process coach. It keeps accountability with the needs of the enterprise as well as the process. The leader / coach steps in and helps redirect a team with a timely word or observation or question to help the team solve problems or redirect when they get stuck or distracted. For example, sometimes, developers are so passionate about solving a problem that they keep going and going, sticking with an approach way too long before trying another alternative. They need a strong coach to step in to help them redirect.
This is a bit of a change from the common approach in Scrum. I am not suggesting that Scrum teams should have managers directly coach them. That would likely be too big a step for many to handle. However, I believe Scrum teams should ensure that they are getting coaching in improving their methods from their ScrumMaster if that is not already taking place. I have seen many ScrumMasters limit themselves to facilitation when a coaching role would be more valuable, even if sometimes a little more confrontational, to the team.
Scrum teams need to recognize and respect management’s need for visibility into the team’s process. Line and product managers at effective organizations are just as committed to the project’s success as the team is. Their questions should be taken as evidence that they need to understand the team’s approach. In his seminal book Kanban (Anderson, 2010), David Anderson describes a case where product managers quickly learned how their asking teams to do “one more thing” adversely affected the teams. They learned to discuss amongst themselves any special requirements for adding something to the mix. This is much more effective than having to tell management being told to do something will require aborting the sprint. The fact that pulling out the aborting sprint card is almost never used is indicative of its ineffectiveness.
Lean is a combination of thinking scientifically and respecting people. Lean is often thought to not attend to the people aspect of software development, but the truth is just the opposite. Respecting people means more than just letting them figure out what to do. It means recognizing that people are human and are therefore emotional. It recognizes that change can be difficult or even traumatic for organizations (6). Kanban is designed as a change management system to enable teams to progress at the pace they are most capable of. This is the ultimate respect, making it ok that people can move at the level they are capable of and not forcing them beyond their abilities.
People learn better in frequent, short steps, than in infrequent, big ones. What I call “Lean learning” would have us focus on small improvements to both our methods and our understanding of these methods. Create explicit policies, test them, and change them… daily if necessary. However, it is also critical that local changes not adversely affect the overall flow of things.
Focusing on coding quickly, at the expense of testing is an infamous example of what not to do. Lean and Kanban focuses on removing delays between steps – typically by either having a small number of items being worked on or by changing the order in which the work is done to minimize delay. The way the work is done is reflected on the project board. This makes the development team’s progress visible. It is usually easy to see how removing one delay affects other steps. This, of course, requires rapid feedback, which is achieved by updating the board in real time.
The Best Way Lean can Help Scrum Teams
So far we’ve discussed how Lean/Kanban practices inside of a Scrum team can be very useful. However, applying lean practices in value stream prior to the development team can have even more impact. In "Lean-Agile Software Development: Achieving Enterprise Agility" (Shalloway, Beaver, & Trott, 2009), I relate a story where an upstream problem in sales was causing a 20% loss of productivity with the teams. Assumptions had been made that poor code quality was the culprit while it was really a challenge with procedures during the sales cycle. Overly focusing on the team often results in missing the big picture.
More typically, however, Lean helps teams by providing business stakeholders and product managers insights on how to better identify, size and prioritize the work to be done by the teams. This should be aimed at reducing the workload teams have to work on at any one time. My experience with large organizations is that the biggest impediment to team productivity is overloading the teams with too much work to do. While Scrum attempts to attend to this by having teams only pull the number of stories from the product backlog they feel they can do during their sprint, reality often has teams be overloaded or working on too many things. Alleviating the upstream pressure often has a very positive effect on the teams by enabling them to focus on the most important work and to have all of the resources they need to get their job done.
It also tends to make the product owners more available to the teams since they are not spending all of their time making a product backlog that is already too large. Working too far ahead on the product backlog often has product owners be unavailable to their teams – further impeding their teams. Virtually every business executive I have talked to has recognized this as a problem when I have pointed it out to them. They understand that development teams need their product managers and product owners available to them while they are working. While they may not understand the cost of delay the certainly understand the cost of not having people available when needed.
By discussing how the work load presented to the team affects the team’s efficiency, executives and management can often be enrolled in being more selective about what truly adds value to the customer. This is often a job too big for the product owner alone since in large organizations many different products are involved.
Thinking about Lean as a combination of science, management and learning provides Scrum practitioners to start with including Lean/Kanban practices into their Scrum practices. Explicit policies, managing work-in-progress, and creating visibility have a direct, measurable impact on a team’s throughput (velocity). By acknowledging there is a set of laws which much be followed, we open up the possibility of conversations between management and teams – where management can both help and understand the work of the teams. Lean suggests learning in smaller steps to validate one’s assumptions by checking the results against the way the team is working, as reflected by the Scrum board.
Lean provides an important and useful environment in which teams can learn more intentionally about their work practice. It makes their decisions explicit. It creates natural and helpful collaborations. And it offers clearer guidance on what to do. Even if many of the external practices of a Scrum team are similar to a Lean-based Scrum team or a team doing Kanban, it is the underlying framework that makes for a more powerful environment.
1. The interested reader should listen to a podcast of mine called Redefining Lean or read Managing the Design Factory (Reinertsen, 1997) which doesn’t mention ‘Lean’ once but is one of the best descriptions of how it works that I’ve ever read.
2. www.wikipedia.com/science, November 2010
3. There is much confusion about predictability and controllability. I recommend my blog Types of Processes – Don Reinertsen for clarification.
4. This is the key tenet of Lean. It transcends manufacturing, services and product development which is why lean-thinking is not limited to manufacturing.
5. I have seen the mere creation of teams – moving away from a matrixed organization – create 200% improvement in true productivity. Why? Because having people work more closely leads to better communication and more focused work at any one time than people working on many things at once with no true teams present. For more information and evidence of this, see my blog post, Challenging Why (not if) Scrum Works.
6. I’m referring to the difficulty many matrixed organizations have creating the true teams that Scrum requires.