Get certified - Transform your world of work today

Close

Comparison of Agile/Scrum Adoption or Maturity Assessment Checklists

Measuring the Agile adoption journey of a telecommunications software product development organization

1 October 2014

Ranjan Ghosh
Independant Consultant


This article was written to share experience I gathered while trying to evaluate ways to measure Agile adoption in an organization. While Internet searches return many options, it took some time to look through them and evaluate their suitability. Sharing my experience will, I hope, put my findings in one place and help fellow Scrum Alliance members who may be similarly tasked to run an evaluation campaign.

As Agile/Scrum practice becomes almost ubiquitous, so has the need to check its adoption and maturity levels. While we have the services of certified Scrum coaches and trainers who carry out the transformation exercise and coach the teams, the onus of the evaluation of the Agile uptake falls upon the quality assessment teams. Scrum trainers and coaches are trained to run assessments, but the organization would have to come up with its own independent assessment eventually. Top management (if one is to use the ISO 9001:2008 jargon) or the sponsor of the Agile adoption would need assurance from time to time that the Agile transformation project is going the way it should. For this they usually depend on the incumbent program or quality management team.

It was exactly this type of situation that I faced a few weeks back. We had to come up with an Agile adoption assessment to track our organization's ongoing transformation. Being a CSPO, I was given this task, as perhaps management felt that with my Scrum Alliance certification I was equipped to give an objective answer. The CSPO study materials, however, did not reveal much except to state that Agile principles would be the prime guiding factor for evaluating a team's Agile adoption. In order for a team and organization to gain from Agile practice, the acceptance had to be more in spirit; otherwise the organization would be in jeopardy and be stuck with dysfunctional practice such as "Scrum-but" or "Agile-fall." Assessing corporate or organizational culture change is not a trivial task, but I felt that if we chose the right assessment method, it would by itself act as a guiding compass, aiding the Scrum team in adopting and adapting.

There is a very good article titled "Maturity Assessment Model for Scrum Teams" by Scrum Alliance member Marmamula Prashanth Kumar. In this article, when Prashanth says, "How do we keep it simple to administer?" he has nailed the issue right on the head. The survey needs to be designed keeping the participants in mind. In our case, since most of the employees are millennials used to short message updates, this factor reigned uppermost in my mind. What I was looking for was a quick, downloadable assessment sheet -- which was not available. (If Scrum Alliance could have a section of tools and downloads, it would be very helpful.)

Then I chanced upon a comprehensive site maintained by Agile practitioner Ben Linders at http://www.benlinders.com/tools/agile-self-assessments/. You may want to visit it and try for yourself a few of the links listed there. I will give my feedback on the ones I visited.

I tried Agile Self-Evaluation by Thoughtworks (which was located at http://agileassessments.thoughtworks.com/online-assessments/agile-self-evaluation but unfortunately no longer seems to be accessible). It was a simple, no-nonsense checklist. The survey questions were apt for practical software development scenarios seen in everyday work life. However, the flip side was that the survey could only be taken by one person at a time, and the assessment result was emailed to that person. So the survey did not scale beyond one or two people's usage. Moreover, Thoughtworks does provide onsite assessment services on commercial terms, so the basic intent is promotion.

The next site I went to was the "Unofficial Scrum Checklist" by Henrik Kinberg, located at http://www.crisp.se/file-uploads/scrum-checklist.pdf. The two-page checklist is available under the Creative Common license (BY,NC,SA) terms and lets others remix, tweak, and build upon the work noncommercially, as long as it is credited to Henrik and licensed under the identical terms. The sheet is very convenient to use. However, one would need a pdf editor to be able to use the soft copy. This checklist would be very amenable for use in a team situation where the ScrumMaster can log in the selection based on simple voting. One can, of course, derive similar sheets using MS Word, MS Excel, or have it hosted on a website under the CC licensing conditions.

If one searches on the term Agile assessment, one result is the "Nokia Test." Upon further research, I learned that the test was first designed by Bas Vodde in 2005, while consulting with Nokia on Agile practice. This assessment was subsequently refined by Jeff Sutherland and renamed the "ScrumButt Test." It covers Scrum practices around iterations, testing, enabling specification, product owners, estimation, burn-down, disruption, and teams. It has a built-in scoring system. Please refer to http://agileconsortium.blogspot.in/2007/12/nokia-test.html for an explanation of the test. What's often considered the original Nokia test can be accessed at http://antoine.vernois.net/scrumbut/?page=intro&lang=en; please see also Bas Vodde's comments at http://blog.odd-e.com/basvodde/2011/02/history-of-nokia-test.html. A word of caution: If you press "submit," the data goes into a common database, so you will not be able to see how you fared. The good part of the survey is that if you are looking for answers about what the current team size and sprint duration are, this is the place to go.

The "Comparative Agility" Agile survey is very comprehensive and well structured. The survey has about 100 questions divided into the areas of teamwork, planning, requirements, technical requirements, quality, culture, knowledge creation, and outcomes. The survey is available in form of a link that can be shared throughout your teams. However, I could not see any place where a consolidation of multiple inputs was shown. So in a way it does not capture the Agile transformation state of the team. Again, as in other surveys, someone has to consolidate the answers of the Scrum team or, if the goal is to measure the organization, have the sample of team members drawn from across the organization give a consolidated response.

The "Open Assessments" test available at the site scrum.org is to facilitate the understanding of Scrum concepts. Based on the assessment tests given on the site, one can gauge the level of understanding of Agile/Scrum concepts. The questions for the online assessment are derived from the Scrum Book of Knowledge. In my opinion, this test is more to aid one's understanding and learning rather to act as an Agile adoption assessment indication.

The site maintained by Ben Linders, where I originally found these links, goes on to list a wide array of other tests, and if one has the time, then a comprehensive comparative study can be undertaken.

It would be ideal if an Agile/Scrum adoption test could be devised and put up on the Scrum Alliance website. This would aid organizations in assessing their Agile practice against a standard. With Agile and Scrum practice gaining in popularity, I feel the time has come to put such a common assessment tool in place. If such a tool were instituted, one would need to ensure that it would allow gauging of maturity assessment based on input from a large sample of organization members, rather than on a single person's responses. This would drive out the bias and allow the correct Agile adoption picture to emerge.


Opinions represent those of the author and not of Scrum Alliance. The sharing of member-contributed content on this site does not imply endorsement of specific Scrum methods or practices beyond those taught by Scrum Alliance Certified Trainers and Coaches.



Article Rating

Current rating: 4.5 (2 ratings)

Comments

Deepak Joshi, CSP,CSM,CSPO, 10/7/2014 2:27:24 AM
Nice analysis for a comparative assessment. I will certainly go through the links you provided.
Koren Stark, CSP,CSM, 2/1/2017 1:43:21 AM
Very helpful! Also suggest checking out Intuit's Agile Maturity Assessment - heard about it from Ian Maple's youtube video.

You must Login or Signup to comment.


The community welcomes feedback that is constructive and supportive, in the spirit of better understanding and implementation of Scrum.


 

Newsletter Sign-Up

Subscribe