Is Documentation Really Wasted Effort?

26 January 2012

Vadivelan Sivanantham
Cognizant Technology Solutions

A widespread myth I've noticed in Agile software development is, "No documentation in Agile" or "Documentation is wasted effort." Particularly during a transition from Waterfall to Agile, we appreciate the benefits of adopting typical Scrum practices, such as short iterations, timeboxing, daily scrums, retrospective, and so on. We also try to get away from the tasks and activities that we found monotonous before Agile adoption — documentation, writing proper code comments, etc. But is it really correct to completely stop documentation and code comments?

Let me share one of my experiences related to this question.

I was the Scrum Master for a team that had just moved to Agile methodology. We were about to make our first release (after three sprints) to production, and we decided to set up a separate meeting to hand over the code base to support team to help them understand the changes (development and support teams were separate entities in that organization). There was a great deal of debate about documentation during that meeting. The support team was expecting some sort of documentation (at least high level) on the architecture and a few comments in the code to help them understand the logic.

The issue was addressed by putting in place the following practices:

  • The Scrum team would hold a detailed knowledge-sharing session with the support team, to go over the new features, any architectural changes, and complex logics in the code.
  • One developer from the Scrum team and one support person would swap roles. This would help the support team operate more independently, with a Scrum team member on board.

Looks logical, doesn't it? The practice worked for more than 15 iterations without any issues. Many features were incrementally added to the product, and there were some code refactoring as well.

However, there was a special business requirement: We needed a specific product for a specific set of customers, a product that customized some specific functionalities and features. After a lot of analysis, the team decided to set up a separate component (a replica of an existing, developed component) with add-on features specific to those customers; it could be plugged into the existing system as needed. Thus we created loosely coupled components and avoided increasing the code complexity of the existing component. A single developer was involved with both the original component and its customized version. Makes sense, doesn't it?

In fact, that component used numerous complex code logics. The developer spent more than a week understanding the code logic he'd written a year before. He did manage to figure out the logic and completed the required changes in the component. And the product was in fact deployed to production after testing — and after resolving a series of post-production defects that the testing identified.

We held a dedicated retrospective on this component and its production. Our key observations and outcomes were as follows:

  1. The developer who developed the component felt guilty that it took him nearly a week to understand the logic he himself had written a year previously. That leads to a lowering of motivation.
  2. To cover that up, the developer put in a great deal of effort (in other words, stressed himself) in order to complete the component customization. This made him to miss a minor logic issue in the code, which led to a number of defects. In other words, the result was less productivity and lower quality.
  3. Our cost-benefit analysis looked at "not including code comments during development" against "time spent in understanding the code logic after a year." We noted that it would have taken the developer ten minutes to write the code comments at the time of development, and that step would have helped him to later understand the logic much more quickly (in perhaps 30 minutes versus one week).
  4. As a result, the team agreed to include minimal documentation and minimal code comments as required in order to avoid this sort of issue in the future.

We learned the hard way that in Agile development, we should have minimal documentation rather than no documentation. The decision about what level of documentation for which components and code logics remains up to the team to decide on a case-by-case basis.

I hope our experience will help others learn the importance of the step more quickly than we did. I would be interested in your own thoughts and experiences related to documentation and Agile.

Article Rating

Current rating: 0 (0 ratings)

Comments

Aniruddha Sharma, CSPO, 1/26/2012 2:38:46 AM
Thanks for sharing your experience. I appreciate your viewpoint and agree to it as well, but the challenge comes when the sprint teams work in continuous delivery mode and they hardly have anytime left for documentation for future reference. For business the documents delivered within scope of a project, are not of much importance as they are attached to the project only till the project end date. Maintenance is not of much concern to them and any efforts invested in this direction is a waste of money for them.

In my project I hardly find any focus on documentation and am really concerned how to convince business for understanding the importance of the same.
Vadivelan Sivanantham, CSP,CSM, 1/26/2012 10:42:16 PM
@Aniruddha - Thanks for your comments.

I hope this experience (which I shared) should help in convincing the business & team on the need for "enough" documentation :-).

Also I think there are couple of points you may have to consider

1. "Sprint teams work in continuous delivery mode and they hardly have anytime left for documentation for future reference" - Documentation should be part of continous delivery. In our scrum teams, we used to have acceptance criteria in 2 levels (DONE-DONE). First DONE is for business acceptance (i.e. functionalities are as per business need), Second DONE if for the team (i.e. components developed are as per the standards, enough code comments, documentations where ever required etc.). The effort required to comply with second DONE should be fairly negligible provided the team is equipped with enough technical experience and standards/Tools.

2. "Maintenance is not of much concern to them and any efforts invested in this direction is a waste of money for them" - This suggests that the business still needs some more coaching and mentoring that they should look out for holistic business value. In our team (including business stakeholders & Product Owner), we always say we have 2 end users for the system (first one being the actual user and the second one is the maintenance team)

Hope this makes sense !!
Sundar Rajan Muthiah, CSM, 1/27/2012 6:12:12 AM
Thanks for sharing the experience!. I believe Agile methodology doesn't suggest that the documentation is not needed or it's not required at all. The real beauty of Agile is that it's flexibility of adopting any kind of documentation which could be based on your business model and the development model. Moreover, the documents (Or effort for documentation) is not considered as part of essential delivery in most projects - that point is well said. I worked in a project which follows Scrum Agile methodology effectively, where the documents are always part of delivery and that was made mandatory for any funtionality development that we do, it could be a new functional development or just a bug fix - as a developer, we will have the responsibility to prepare as much documents as possible. And, it's mandatory that we should put as much comments as possible for each line of code that is written. That way, it helped us a lot during a transition to supporting team and for the reverse engineering of futur enhancements even after the team member who developed that particular component left the project.
Mathan Thillaiyapillai, CSM, 1/27/2012 2:05:04 PM
Couple of points.
1. In Agile practices (remember it is NOT a methodology), the "no detailed documentation" means no upfront detailed requirement and design documentation. It comes from the fact that Agile emphasis customer interaction over contract negotiation (complete requirement documentation). If a customer or product owner hands over a requirement document, it implies there will be less engagement between customer and development team. Also, customer has thought through the entire project/product requirement (or at least he/she think so) customer will not benefit from agile teamΓÇÖs willingness to accommodate change.
2. Adding comments to code is not documentation, it is a mandatory coding standard. In fact if there is no proper coding, then it perfectly reasonable for Scrum Master to bring that to the team and may recommend that the User Story in not complete. (or bring this up in retrospective )
3. DONE, DONE or DONE, DONE, DONE should be strongly discouraged. There has to be one and only definition of Done.

You must Login or Signup to comment.