How Strategic Management Processes Can Imitate Agile and Scrum

19 December 2013

Alistair Davidson
Eclicktick Consulting

Laura Klemme
Klemme Engineering


Executive summary

There are parallels between the evolution of strategic planning into strategic management and the evolution of Agile best practices. Success with Agile and Scrum often requires understanding the current strategic management practices within the organization. Top-down, command-oriented organizations still predominantly doing centralized strategic planning are typically less naturally receptive to Agile development. Strategic management-oriented organizations may not only be more receptive to Agile/Scrum development but may also discover that the example of the well-specified methods of Scrum offers opportunities to improve the ways that strategic management-based planning and product management planning are conducted.

Introduction

In the early days of strategic planning, "brilliant" MBAs using the latest strategic theories developed business strategies for others to implement. Most failed. It turned out that telling individuals what to do is much less effective than involving those who must implement strategies in the development and execution of strategies. In a sense, it was not a new insight; divisionalization and the use of strategic business units (SBUs) that mimicked the structure of a full business were also an attempt to push down accountability and responsibility to the execution levels in the organization. It also turned out that the 40,000-foot view of strategy could not succeed without successful on-the-ground execution, knowledge, learning, and adjustment. Such on-the-ground execution required the commitment and passion of the organization rather than merely writing a superficial plan.

In software, similar lessons have been learned. Software is difficult to get right. What you specify up front always seems to need to evolve as the understanding of a problem and users improves or as markets and competition change. "Brilliant" and often overconfident managers (typically engineers, MBAs, and highly educated technocrats) specify requirements and toss the specifications over the wall to the developers. They, in turn, deliver untested code to test departments. Users are last in the loop. Managers who are distant from the development process, lack understanding of the technical choices, and have underinvested in customer involvement/feedback/sponsorship use their power to make project and specification modifications with a 40,000-foot view of the project. Their decisions are typically more affected their status and role than by data and knowledge.

While the parallels between the evolution of strategy and the evolution of best software practices are not 100 percent, there are significant similarities between the development of strategy and software practices.

  Strategy: Strategic Planning Strategic Management Software Development: Waterfall  Agile
Early management approach
  • Top down.
  • Based on largely theoretical models.
  • Separation between design and execution of strategies.
  • Iterative learning not handled well.
  • Lengthy implementation cycle before learning and improvement can occur.
  • Learning occurs at the execution level rather than the planning level.
  • Waterfall method of development.
  • Design based on incomplete knowledge of requirements.
  • Separation of design from implementation from testing.
  • Lengthy development cycle before learning and improvement can occur.
  • Learning typically occurs across project staff with different responsibilities and few mechanisms for integration of learning.
Drivers of intellectual shift
  • Widespread dissemination of management knowledge due to increase in number of business schools and MBAs.
  • Emergence of object-oriented (OO) or a modular approach to software development.
  • Widespread adoption of OO development.
  • Lowered cost of design reuse, code reuse, and testing.
  • Emergence of purchasable class libraries that provided close to plug-and-play functionality.
Second- generation approach
  • Push-down of responsibility for developing and executing strategy to those who are responsible for implementing.
  • Cascading of accountabilities, strategy development and execution planning via facilitated workshops.
  • Recognition that large projects are difficult and that iterative development provides more opportunities to validate software design, user interfaces, and user needs.
Third- generation approach
  • Strategy is largely subsumed into normal business planning processes.
  • Strategy models become part of the vocabulary of many line managers.
  • Wide variation in the understanding and successful implementation of models.
  • Software design and development is reconceptualized as knowledge work where the challenge is process improvement rather than construction of software.
  • Design and testing are no longer separate from implementation.
  • Wide variation in adoption and effectiveness.
Organizational resistance
  • Driven largely by "baronial" resistance to redistribution of power in the organization.
  • Dysfunctional bureaucratic incentives.
  • Reduced risk taking by risk-averse employees in a less friendly environment.
  • Control-oriented bureaucrats are uncomfortable with loss of power over development projects.
  • Scrum team members are punished for challenging the status quo and there are few incentives for seeking improvement, leading to lowered rates of learning.
  • Managers with high need for power prefer to hire younger and less experienced workers, contract workers, and/or H1B visa workers whose inexperience/insecurity makes it less likely that they will complain, take risks, or challenge traditionally oriented leaders.
Successful organizations
  • Treat employees as assets
  • Reward employees for risk taking and activities that create learning and improvement.
  • Expertise is explicitly tracked and managed.
  • Focus on creating value for customers and customizing relationships, co-creation of value with customers.
  • Leadership communicates its commitment to empowering knowledge workers.
  • Education and empowerment are supported.
  • Older workers' knowledge is valued more highly.
Competitive implications
  • De-bureaucratized organizations move more quickly to develop innovations, improved processes, custom mass production, and gain market share.
  • Improved and modular development processes increase the rate of improvement in value creation.
  • More sophisticated approaches to architecture and data analysis enable reconfigurable software, persona-driven development, individualized marketing.

What makes the interaction of strategic management and Agile/Scrum practices so important is that in fast-moving markets, better development practices enable faster learning and superior return on capital. New technologies and developing leadership products enable premium pricing, descending learning curves faster, faster creation of ecosystems, new strategies, new types of value added, and new business models. However, in comparison to the well-described methods laid out by many authors writing about Agile and Scrum, there is a marked absence of similar specificity in the strategic management literature and in business practices. For example, in Scrum we operate at three levels of planning: the daily stand-up meeting, which reports on daily activities; the Sprint planning and implementation cycle, which typically runs from 2 to 4 weeks; and the longer-term 4-to-12-month release planning. Only the longer-term, generational architecture planning aspect of software development is less well addressed.

In contrast, strategic management execution processes are much less well specified. Strategic management push-down, where objectives are pushed down to operating groups for more detailed planning and to obtain commitment, are typically poorly described by managers, writers, and researchers. It is a surprising conclusion when such processes go back to the early 1980s, ten years before the Agile Manifesto. A much worse problem is that strategic management is typically done once a year -- essentially the strategic management process is fundamentally driven off the annual budget requirement rather than being driven by the requirements for development, learning, and implementation. Only a few companies do flexible budgeting/strategic planning/strategic management exercises as needs evolve.

Strategic management software tools first began being deployed in the middle 1980s. The first generation of tools included expert systems for assessing competitive strategies (e.g., Alacrity Strategy), business plan writing tools (e.g., Advia Decide and Manage), portfolio analysis tools (for graphing bubble charts), regression-based tools for modeling predicted new product success (e.g., NewProd/Danprod), and strategic cost driver tools that modeled the impact of scale, scope, experience curves, and complexity. Many of these model-driven tools are no longer available or no longer popular. More recently, a variety of new, often SaaS tools for cascading objectives and tasks down the organization have become popular. But, unlike Scrum, these tools are often used in a demotivating way. Having a senior manager be able to drill down into multiple layers of the organization and/or complex projects often devalues the autonomy of innovators, product owners, ScrumMasters, and developers. The military, which has grappled with span of control issues for decades, often has rules about the visibility of data to managers high up in the organization in order to avoid emasculation of lower-level officers.

An important reason for being Agile

Rigidly thinking, command-oriented managers tend to see employees generally, and developers specifically, as "cogs" rather than as human beings whose rate of learning, knowledge, passion, and motivation make a significant difference. They forget that employees/developers learn and improve, and, when in teams and working with the right culture and incentives, learn from each other.

In a sense, the problem is similar to that of the American automobile industry in the 1970s. Cars were designed and designs were "tossed over the wall" to production engineering. Cars were produced on the assembly line and quality problems were caught too late -- after the car had been built. And there was no systematic capture of the knowledge of assembly line workers, so the opportunities for incremental improvement were not being captured. The Japanese improved their automobile product process faster than American firms with extensive quality training, by appointing product owners and giving them authority and power, and by tapping into the learning and insights of the entire organization, down to the factory line worker. Harnessing the frontline learning of retail staff is now one of the leading areas in retailing, representing a similar trend.

Scrum: Reframing the problem, simplifying processes

The way out of the box for both strategic improvement and software development is to be humble and lazy. Humble means not assuming you know what the customer wants. Lazy means being ruthless about what you develop. We like to call this approach identifying the Pareto features of a product, i.e., the 20 percent or so of features that were critical to a customer being able to use the product so that you could get feedback. Others have called it iterative or Lean development, the minimal viable product (MVP) or minimal marketable product (MMP) -- but whatever the name, the strategy is always to develop quickly and get feedback quickly. And you need to close the loop and continually improve not just the product or project but the way you develop and learn about markets. This rapid iteration rarely matches annual planning cycles.

One of the least-discussed and most important motivating reasons for Agile processes is the unpleasantness of the software developer experience, which is often paralleled in organizations unsympathetic to Agile autonomous teams in knowledge-based organizations . Ed Yourdan summarized the problem in the title of one of his books, Death March. Bad practices in software development have made it normal for managers to expect that the way out of trading off between time, quality, and cost of projects was to incur overtime. As a "good" manager, you demanded that programmers work 7-day weeks and 14-hour days. A "bad" manager would fail to deliver on time by not demanding long hours of developers and testers. Working 80-hour weeks is not typically conducive to high performance. Even if it is possible for a short period of time, because you are passionate about a personal project, it is not sustainable over the medium term let alone over a career. The result is that only naïve young programmers are likely to put up with "death march" projects, and only for a short period of time. So there are five likely consequences: bad code, programmer burn-out, not being an employer of choice, lost knowledge and resulting lost opportunities combined with higher rates of mistakes, and higher costs of on-boarding a "revolving door" of employees.

Encouraging revolution

If a company is fortunate, the spread of Lean, Agile, and Scrum can be viral, as best practices spread within an organization, though without the support of external stakeholders, these approaches will be less successful than they would otherwise be. Research on autonomous change teams, a business situation similar to Scrum, suggests that successful autonomous teams have three basic ingredients for success:
  1. Organization-wide support for the autonomous (i.e., Scrum) team
  2. Unwavering support for the resources needed by the autonomous team
  3. Frequent communication between the team and leaders in the organization
There is perhaps another way that improved Agile and software processes may emerge, and that is a form of "industrial action." It may be subtle in the form of change and development teams being discontented and demanding change, or it may be triggered by dissatisfaction that produces employee turnover. Perhaps less intuitively, companies should hope for revolution by developers, innovators, and product owners, because the alternative is the leakage of talent, leaving a company with the untrained, the inexperienced, and the mediocre. Ironically, the more that a company is in need of revolution, the more likely it is that the controlling "command" management style will be perceived as punishing innovation and process change, with the resultant exit of those who could improve performance.

Strategy determines innovation strategy: Innovations enable new strategies

Strategy is rarely the result of one insight from a senior manager: Strategy, incremental improvement, operational improvement, and innovation typically come from the aggregate knowledge, learning, skills, and experiments throughout a firm. The process of managing these collective insights all the way from observation to analysis to experiments to commitment to improvement represents the strategic management process in its broadest sense.

Strategy is enabled by faster learning. Innovation, done faster and well, creates competitive advantage. And the selection of technologies, technology goals, and technology resource allocation should be driven by the strategic vision, mission, values, business line, and product strategies. When you can automate anything, you need to know what you stand for and what your goals are. But the corollary is also true: Technology enables new possibilities and new economics. Faster and more effective deployment of technology may open opportunities that reveal new potential strategies, confirm strategic hunches, or expand upon initial small experiments.

There is a different between stubbornness and persistence in both strategic management and technology development. Stubbornness is about believing that you are right without having strong reasons that support your belief. Persistence is about selecting an objective and adapting, e.g., in each new sprint as new information is acquired and evaluated.

Five ways that Scrum can influence strategic management processes and product management processes

  1. Companies need to have the equivalent of an Agile Manifesto, perhaps a Strategic Management or Product Management Manifesto to empower product owners and autonomous work teams. The goal is to prevent innovations from being nibbled to death by ducks, i.e., having too many chefs in the kitchen.
  2. Innovation does not follow a highly detailed annual planning cycle, just as much as it does not follow a Waterfall development approach. Organizations need to formally address the use of Agile innovation and development processes with different processes than the annual budget/strategic management approaches.
  3. Planning granularity for innovators should be different than the planning granularity for larger, more predictable businesses.
  4. Planning and execution processes need to be matched with what Roger Martin classifies, in The Design of Business, as intuitive innovation, heuristic innovation, and rule-governed innovation. Early-stage businesses are often driven by intuitions about opportunities. Slightly more mature innovations may be characterized by some rules of thumb or heuristics about a market or opportunity. Well-established businesses often have detailed models and rules about how the business works.
  5. Education of nontechnical managers about Agile development, its superior productivity, its ability to deliver higher value faster, is required to have supporting stakeholders in the organizations for Agile innovation and Agile software development. It may encourage them to think differently about product releases and the business models that take the most advantage of fast learning -- e.g., Adobe's recent switch to a subscription model for its key products.

Further reading and references
  • The Agile Manifesto.
  • Beck, Kent, and Andres C. Extreme Programming Explained: Embrace Change, Addison-Wesley, 2004 (second edition).
  • Davidson A. Innovation Zeitgeist: Digital Business Transformation in a World of Too Many Competitors. Eclicktick Consulting Kindle ebook. 2013.
  • Davidson A. Scrum Master or Scum Master: How to Build or Destroy the Motivation of Your Team. Eclicktick Consulting White Paper, http://www.eclicktick.com/scumalliance/.
  • Hess J. Empowering Autonomous Teams. Ivey Business Journal. Nov./Dec. 2013.
  • Martin R. The Design of Business: Why Design Thinking Is the Next Competitive Advantage. Harvard Business Review Press, 2009.
  • Peters L. The Peter Principle. Buccaneer Books. 1996 (first published 1969).
  • Rubin K. Essential Scrum: A Practical Guide to the Most Popular Agile Process. Addison-Wesley, 2012.
  • The Scrum Manifesto: http://www.scrumalliance.org/code-of-ethics.
  • Yourdan E. Death March. Prentice Hall, 2003 (second edition).


Opinions represent those of the author and not of Scrum Alliance. The sharing of member-contributed content on this site does not imply endorsement of specific Scrum methods or practices beyond those taught by Scrum Alliance Certified Trainers and Coaches.



Article Rating

Current rating: 4.2 (6 ratings)

Comments

Be the first to add a comment...


You must Login or Signup to comment.