Whenever an organization engages in Agile, it’s usually the case that someone is called in to do an Agile assessment. This typically involves setting up a series of interviews and looking at documentation, with the aid of various questionnaires, online instruments, and spreadsheets that generate numbers for management to understand where the organization stands with respect to agility.
The challenges of using questionnaires
Using questionnaires presents two challenges. First, while the questionnaires often include organizational components, they are usually limited to the degree of business engagement and leadership support. They will often not include other salient organizational issues that are potential risks or opportunities for Agile adoption. The way work really gets done in organizations often has little to do with the boxes and arrows in the organizational chart, and a lot to do with the white space between those. It’s a case of formal and informal culture, and questionnaires are not particularly sensitive to those dynamics.
Second, the use of the questionnaires can easily limit the perspective of the interviewers, as a series of checklist items and the focus on completing the list can inadvertently distract assessors from other, more subtle issues, such as interactions across work units, cultural norms that don’t fit into the instrument, and the like. While this isn’t always the case, as the quality of the assessment depends on the experience of the assessment team, the tendency to "stick to the program" is reinforced by the existence and use of a set of predefined questions based on unarticulated assumptions (that may or may not be true).
Using the Whole Systems model
We’ve been working with another approach to Agile assessments using a "whole systems" model. Since Agile, when fully implemented, is an ecosystem across many different aspects of an enterprise, we thought it would be useful to look beyond the team — or even the program — to better understand the organizational dynamics that would present challenges or leverage points for Agile adoption.
We’re using a model adapted from Mary Gelinas and Roger James, which they use for their consulting practice for organization redesign. In our experience, the model can be extended from team to enterprise by shifting the labels in the Venn diagram below. A sample program-level view is shown here, with some focus on governance in the program as part of the larger enterprise governance. The values for the center circle and the outer loop could easily have been "Mission, Vision, Values," or "Leadership," or other factors that reside at multiple levels in the enterprise. The Venn diagram enables a view of the scope (and potential boundaries) of the assessment. It also underscores the intersections where the action is happening.
Applying the Grounded Theory method
In conducting the assessment, we use an informal variant of the Grounded Theory
method, which is an inductive approach that reduces the tendency to draw premature conclusions stemming from a priori
hypotheses. This enables us to start with open-ended questions that lead to other questions, and then to others, so that over time we develop a picture of the organizational context for Agile adoption. In the process, we follow all the standard processes, from reviewing documentation to interviewing at all levels of the organization.
In our work, we develop a rich source of data across the enterprise that provides an observational foundation for developing hypotheses to further test and validate. In some cases, we code our data; in others, we simply review and articulate the patterns of data using the model as a reference point. From our perspective, observations cannot develop into findings — and eventually recommendations — without triangulation of the data across at least three different data sources, preferably at different levels and organizational units. We triangulate at observation and finding levels, such that findings are summary statements of an integrated set of observations or lower-level findings.
This approach doesn’t eliminate the use of questionnaires or other assessment instruments. On the contrary, these tools can provide a numerical perspective that enables leadership to judge the level of Agile readiness, risks to adoption, and other factors. Although this is useful as a metric, it doesn’t always provide the depth of understanding or identify the challenges to adoption needed for effective decision making and Agile strategy development.
We have found that leadership wants more than numbers; it wants answers and direction. The rich source of data that we gather provides the base for substantive recommendations, which tooling by itself can do only with inferential judgments. And we have found that the substantive data set we uncover provides a credible base for useful and targeted recommendations.
Once we’ve created our recommendations, we use a modified Cost of Delay and Weighted Shortest Job First spreadsheet to establish our preliminary determination of critical issues to be resolved, and in what order. We then work with the client to validate our assumptions and change the priorities based on their deeper understanding of the organization. From there, we use story mapping with the client to develop their Agile adoption road map.
This approach takes us on average about three to four weeks with two assessors. Two assessors are useful as cross-checks against individual bias, as well as providing a better set of experiences from which to define recommendations. We’ve field-tested this approach with two clients, both with good results and excellent feedback.
This approach does require some experience in "working in the shadows" of the organization to identify and understand the social and political dynamics in play, as well as the more evident practices, training, and structure that Agile demands. In that sense, it’s somewhat consultant-dependent, but any Agile coach can take this approach and learn more than we typically do in assessments that are based on only questionnaires and tools.
So give the Whole Systems method a try, learn from it, and adapt it to your needs. And let us know how it’s working for you.
Note: This article was coauthored by Bill Bear.