At a recent Scrum gathering, Ken Schwaber and Jeff Sutherland spoke about technical and design death. Although the concepts themselves are not new, Ken and Jeff have identified a number of characteristics that (I believe) are original insights. In addition, they also outlined some useful graphs that a company or department could use to indicate the "health" of their software products.
I was originally going to write an article that discussed what these graphs indicated and the data required to construct them. In order to do that, though, I wanted to reference articles that illustrated Ken and Jeff's ideas. I searched the web without much success. Finally, I decided that I needed to write both an introduction and a follow-up article. This first article in a series of two is the introduction and discusses the concepts of technical debt, design death, and their telltale characteristics.
Over the course of a project, it is tempting for an organization to become lax regarding software quality. This may be for a number of reasons: the primary ones being that the team is expected to complete too much functionality in the given time; or quality is not considered a high priority characteristic of the software.
Let us consider a contrived example with a mythical “average” project team that’s currently working on the company's flagship product (See Figure 1). Let’s assume that they are not making as much progress as originally envisioned (See Figure 2).
Figure 1: Expectations at the start of a project.
Figure 2: Reality has a way of making its presence known!
Senior management will want to know why the team’s progress has not met expectations. Subtle pressure is brought to bear in order to bring the project in on time (see Figure 3). The project manager explains the new timeframe by saying that “the project team has learned to be more efficient.” This is not really true. In order to come in on time, the project team reduced the quality of the software. The difference between what was delivered and what should have been delivered is known as technical debt (see Figure 4).
Figure 3: Pressure is placed on the project team “to be more efficient.”
Figure 4: Technical debt.
Now let’s consider the next project built on the same code base. The new team has exactly the same issues with regard to delivering software on time, but this time they also have the additional disadvantage of working in a code base that is of a poor quality. Their progress is even slower than the first team’s. Again the same management techniques that worked so well last time are used and pressure is again brought to bear on the project team. They finish "on time and within budget" but weren't nearly as “efficient” as the first team. What’s happening now is that a layer of cruft is built upon a layer of cruft (see Figure 5).
Cruft (n): 1) An unpleasant substance. The dust that gathers under your bed is cruft; the TMRC Dictionary correctly noted that attacking it with a broom only produces more. 2) The results of shoddy construction.--Jargon File 4.2.0
Figure 5: Cruft on top of cruft.
A Bad Case Of Cruft
The concept of software complexity as debt was originally coined by Ward Cunningham in an experience report for OOPSLA ‘92. It has since gained a large following and has been expanded to include technical debt and design debt in addition to other concepts.
Technical debt is simply deferred work not directly related to new functionality but necessary for the overall quality of the system. Examples of this include delaying upgrades to necessary tools and frameworks, delaying the refactoring of overly complex system components, and so on. If this technical work is necessary for the health of the system, then what happens when it’s ignored for too long? What happens when there are several layers of cruft and no unit tests to aid refactoring?
Fast forward two years; the company’s flagship product is now on its fourth version. It has significantly expanded in scope since the original, and is very, very complicated. So complicated, in fact, that there are sections of the code that only one or two developers in the entire company have the ability to make changes to. Classes may be several thousand lines of code.
True story: I was recently with clients who described their classes as being very complicated. I asked them, “How long are your classes? Three or four thousand lines?” The client staff all burst into laughter. I was puzzled by this so I asked what was so funny. “We're lucky if our classes are three of four thousand lines ... try ten thousand lines, plus!”
When code reaches this level of debt, the effort required to change any part of the code is significant and for all intents and purposes the code can be described as design dead. Ken has identified three attributes of code that has entered design death:
- The code is considered part of a core or legacy system. The functionality is interconnected to so many other parts of the system that it’s impossible to isolate (and hence reduce) any one component.
- There is either no testing or minimal testing surrounding the code. This is really tautology by implication. Without comprehensive unit tests it is impossible to refactor the code to a more manageable state.
- There is highly compartmentalized knowledge regarding the core/legacy system, and it may be supported by only one or two people in the company.
Knowing all this information is all very interesting, but how does it help a company? How does this information help companies that have a significant number of legacy systems? What options does it provide them? That was the subject that I originally set out to discuss and will now be written in part two.