"Observe always that everything is the result of change, and get used to thinking that there is nothing Nature loves so well as to change existing forms and make new ones like them."
-- Marcus Aurelius
This paper, originally written in February 2013, brings to light some of the least-discussed topics and consequences of "broadband agilization" that currently take place in the industry. The materials of this paper are subdivided into two general sections:
The first section describes certain impacts that Agile has on individuals and their personal career advancements.
The second section describes organizational-level Agile impacts that pertain more to client companies that undergo Agile transformation, as well as service-providing vendor companies that deliver Agile-transforming expertise to their respective clients.
The reader will most likely focus on the section that best represents his primary interests and concerns. However, it is recommended that both sections are read in full, as in unison they create a better holistic perspective of the industry changes brought about by Agile-mania.
The reader will be taken out of his comfort zone and forced to think more uninhibitedly and realistically about those aspects of Agile that may not be as obvious and are not explicitly covered in other literature.
Organizational impact of Agile
"Change does not necessarily assure progress, but progress implacably requires change. Education is essential to change, for education creates both new wants and the ability to satisfy them."
-- Henry Steele Commager
According to Wikipedia (http://en.wikipedia.org/wiki/Peer_pressure
), peer pressure
is defined as "influence that a peer group . . . exerts that encourages others to change their attitudes, values, or behaviors to conform the group norms."
Image courtesy of http://www.markstivers.com
Today companies often decide to introduce Agile practices without thoroughly thinking through why they are doing it, without doing enough due diligence and research to reasonably attest that, indeed, their efforts will bring benefits in the long run. Instead, these companies do so because their executive management has decided that "the time has come," since there are so many other peers out there that do the same.
For such companies, Agile adoption has almost taken the form of a fashion statement, a way to prove to themselves that they are up to date with others. Such companies care more about keeping up with the mainstream than making a well thought-through and carefully planned step forward of bringing better values, practices, behaviors, and cultural patterns inside their walls.
There is whole array of commonsense Agile transformation readiness prerequisites that these companies either ignorantly overlook or intentionally ignore. These companies go after what may seem to be a very good cause. However, it is nothing more but a chase of the “status quo.”
In majority of cases, such poorly substantiated motives for Agile adoption bring about failure, and since there is only one chance to make a first impression, any future attempts to reintroduce Agile at a later point, even when conditions might become more favorable, usually meet resistance and very little support. Trust is lost as nobody wants to repeat the same mistakes twice.
"Going Agile" should never be a final goal. Agile is just a way to get to something much more measurable and tangible. For example, achieving near-term and long-term economic benefits, ensuring cost-effectiveness and higher return on investments (ROI), adjusting corporate cultures and working environments in ways that make companies a more desirable place to work.
Many firms proudly announce that they have been undergoing Agile transformation without accepting the fact that in order to truly appreciate the benefits of Agile tools and techniques as the mechanism for more effective product development, firms also must adopt agility in their structure and culture. The former is just not possible without the latter.
Specifically, if an organization does not have in its plans to fundamentally adjust its conventional hierarchical structure, flatten its convoluted reporting lines, remove redundant roles that do not really contribute much value to the overall process, and trim wasteful activities, then there will be no cultural shift.
While using the Toyota Production System (TPS) as an example in their studies of Lean, Tom and Mary Poppendieck have clearly identified seven types of waste in Agile Product Development.3
The third item on their list of wasteful activities is "Extra Processing." If we think about any process in terms of how many people are involved in it, then it begs the question: Should removal of wasteful processing also remove wasteful processor(s)?
Just as an assembly line automation makes a lot of manual labor unnecessary, eliminating unnecessary steps in a process makes the performers responsible for those steps also unnecessary.
This supports the claim that in order to successfully transition to Agile, companies should be willing and ready to reorganize their cohorts in ways that require trimming off what is no longer needed.
In his writings, Craig Larman5
alludes to organizational waste management by referencing not just wasteful and counterproductive processes and individual behavioral patterns but also certain organizational layers and individuals that cause retardation of the Agile process. While being a big proponent of flattening organizations in general, he explicitly refers to certain mid-level management that should be removed due to their lack of value.
There are some other pitfalls that are more frequently seen with larger, enterprise-level organizations:
Inability to perform cost/benefit analysis to determine whether the adoption of an Agile framework is actually monetarily beneficial for a company. Since larger companies typically do not release to production as frequently as smaller ones, it is not always easy to map end-client satisfaction and demand, and subsequent revenue increase to changes of product development approaches. This delayed cause-and-effect is due to longer time to market and prevents executive management from seeing results of Agile transformation soon enough to decide whether it is worth continuing the experiment. Executives do not use a "stop the ship" approach to objectively analyze what has been accomplished in order to decide whether it makes sense to continue. If things do go south, by the time executives realize this, the damage is too high to be dealt with silently, behind the scenes. When the political current gets too strong, as it does in majority of cases, fighting it and acknowledging that the latest and greatest add-on to a company's strategy is not working is not something that the top echelon of executives can do easily.
Inability to take into account existing business relationships with third parties whose operational models, processes, and functions adversely impact Agile adoption. Specifically, dependencies on third-party product development vendors who do not practice Agile, do not deliver incrementally, and -- what is even more alarming -- fundamentally are not equipped for transparent two-way communication (they prefer everything in writing, sealed and signed) is very costly. This can negatively impact a company's ability to produce its own deliverables. Such relationships and dependencies must be thoroughly reevaluated and, if necessary, terminated.
Inability to develop standardized techniques or mechanisms to measure levels of Agile maturity across multiple organizational verticals. Although developing universal best practices for multiple teams, even within the same organization, is not expected (it would also contradict the idea of decentralized control and frequent inspection and adaptation -- something that is required by Scrum), the ability to tie low-level (local, single-team) Agile metrics to global performance indicators is possible and even desirable, as ultimately every company measures its profit and loss by using universal units (currency).
Instead of attempting to do "big bang," top-down Agile transformations of multiple teams, projects, and programs at the same time, it is much more advisable for companies to start small, to consider a pilot Agile project to gain small, quick wins, and then gradually proceed with Agile adoption by sharing knowledge and lessons learned laterally: from team to team, from project to project. It is important, however, that executive support and buy-in come all the way from the top-executive level -- what is inevitably required for a lasting cultural shift. Executive-level coaching is required for this to be a success.
Numbers do lie
"Nothing in education is so astonishing as the amount of ignorance it accumulates in the form of inert facts." -- Henry Brooks Adams
In most of reference literature, the word "metric" is defined as the system of standard or measurement. In the conventional world, the notion of metrics analysis is frequently associated with establishing success/failure criteria, progress indicators, and benchmarks.
Although the ability to properly analyze and communicate Agile metrics data still serves its meaningful purpose for Agile teams, as it gauges them in their journey toward maturity, for large-scale Agile transformations at enterprise level (no so much small- and mid-size organizations), the improper use of Agile numbers is common and dangerous. The ability to incorporate Agile data into a broader picture and integrate it with other enterprise-level measurement tools, techniques, and analytical facts is frequently lacking.
Image courtesy of http://www.processexcellencenetwork.com
If Agile transformation is driven from the top of an organization but appropriate training is not timely provided to executives, then their expectations are not properly set, and this leads to misjudgment and poor decisions.
The problems vary and include constant pressure on workers and deterioration of morale, mistakenly (in a rush) selected corrective actions, and/or making things "look pretty" by falsely communicating unachieved progress further up the chain of command and to the rest of organization in order to preserve personal credibility and reputation.
One of the most common misinterpretations is of metrics obtained from Agile collaboration tools. The words of wisdom in IT are: "A fool with a tool is still a fool." This wisdom has proven itself many times. One of the most frequently misinterpreted and misused metrical units is velocity -- specifically, what it measures and what it depends on.
Here are some most common misjudgments
Velocity reflects how much time it takes for a team to complete work. No consideration is given to the fact that in order for complexity estimation to be accurately translated to time, the same team must point stories and stories must come from the same PO, who has the same writing style and is able to write "sizably." This is wrong.
Without prior normalization of story point estimation across multiple teams (deriving a conversion factor to normalize 1 point of one team against 1 point of another team), it is OK to compare velocity of one team to that of another team. This is wrong.
Over time, velocity should continuously increase and if it does not, it indicates that a team has stopped improving. This is wrong.
A team's velocity can be increased by increasing the number of team members; this is an effective way to increase velocity. This is wrong.
Resource rotation (on/off a team) to ensure cross-training and knowledge transfer outweighs the importance of keeping a team together and does not have impact on velocity. This is wrong.
Reestimating work mid-sprint is acceptable, if it can provide a correction to inaccurate estimations at the beginning of a sprint. This is wrong.
Attributing a certain percentage of a team's committed velocity to an individual team member, based on the contribution of that individual to any particular story, is an objective way to measure individual productivity/performance and a reliable way to gauge overall team velocity. This is wrong.
This is how some of these false interpretations come about:
Figure 1: Comparing un-normalized velocity
Figure 1 above illustrates an example of three separate teams being compared against each other in terms of their velocity. Such a comparison does not hold value for the following reasons:
Team size and, subsequently, its capacity, most likely varies. This means that the man-hours of one team are not comparable to the man-hours of another team.
The estimation scale of each team is different. In order to compare a story point of one team with that of another, a conversion factor must be derived to understand what each team's story point really means in terms of ideal hours.
Finally, Agile maturity of teams might be different. Although a team's maturity can be related to a team's productivity/output, comparing teams that are novice to Agile with those that have been in operation for a while and have developed some cadence is not objective.
Here are some graphic illustrations of how such improper judgments originate:
Figure 2 below illustrates an example of a false expectation that, over time, the velocity of each team will indefinitely increase. Team 2 (brown) appears to have the velocity trend that aims at infinity. Team 1 (blue), on the other hand, has reached a plateau and is no longer increasing. It is false to assume that the velocity trend of Team 2 is better than the velocity of Team 1. Yet, frequently, this is exactly the conclusion management derives when they are presented with multi-team velocity trend charts.
Figure 2: Expecting continued velocity increase
Such ever-growing to supersonic velocity of Team 2 is not realistic. Increasing velocity to a certain reasonable level and then sustaining it would be much more reasonable. Having a steady velocity would also make forecasting and forward planning more reliable.
Figures 3a and 3b below illustrate how a team's sprint velocity can be mistakenly attributed to individual contribution.
Figure 3a: Measuring "individual velocity"
Note: If added diagonally and horizontally, percentages add up to "1" (100%)
Figure 3a shows how three individual team members spend varying amounts of time (in percent) on each individual story (later, accepted). Such work distribution could be very reasonable as each individual may need to contribute differently to each story, based on his type of functional expertise and type of work required for each story.
Figure 3b, on the other hand, shows a very unreasonable calculation and, subsequently, a conclusion that is frequently made by conventionally thinking management, especially if an environment strongly supports the idea of individual performance. According to this figure, each team member is assigned a certain amount (even fractional) of story points, based on his percentage of effort contribution to each user story. The numbers are derived by multiplying individual percentage contribution and number of story points each accepted user story is worth. For example, John, who performed 0.45 (45%) of work (in hours) against Story A (3 story points), established his own “velocity” of 1.35 story points against Story A.
Figure 3b (Measuring “individual velocity”)
Such an approach is wrong, and here is why:
To achieve higher productivity and cohesiveness, Scrum team members are expected to swarm
-- work collectively on the same story or sometimes even a single technical task to achieve common success. Linking any single piece of a team's work, successful or unsuccessful, to individual contribution and performance is inappropriate and subjective: it prevents teaming and collaboration as well as makes team members worry about individual achievements more than about overall team success.
Another reason why the above described calculation is a fallacy is that although each team member cumulatively contributes the same amount of time to sprint work, based on an individual's functional skill set, each person may spend time (capacity) differently against differently pointed stories, which means that the multiplication factor in deriving individual velocity is not constant from the beginning.
Another misuse of the velocity concept is splitting up partially done stories at the end of a sprint to produce "partial" or "conditional" acceptance by the PO. By doing so, higher team velocity gets fabricated.
While breaking the concept that each story must be an independent and deployable piece of functionality with intrinsic business value, such numbers-cooking and story-point chasing will prevent a team from establishing a reliable velocity and doing accurate sprint forecasting and strategic planning.
There are also some other inaccurate interpretations of Agile metrics that, for the most part, have to do with leaders' lack of understanding of the economic principles behind product development. Among others mistakes, the two that are very commonly observed are around Capacity Utilization and Work in Progress (WIP).
For example, forcing teams to maximize their Capacity Utilization by increasing individual and team workload to close to 100 percent, as it is frequently seen in "high-performance" organizations, especially when using offshore resources, is a fallacy.
Such an approach forces teams into working without any slack time and, therefore, deprives teams of any chance to improve processes. By the same token, pushing teams into making aggressive commitments during planning and starting a sprint (e.g., in Scrum) with an initially unrealistic amount of work causes end-of-sprint failures. All of this results in a team's diluted focus, excessive multitasking, making irrational decisions, and ultimately producing code of very low quality.
Expecting high Work in Progress (WIP) by having executives constantly question teams about the amount of work in flight is yet another fallacy.
By applying such orthodox beliefs that everyone should be preoccupied with their own work at all times, and promoting the idea that a high amount of work in progress
is a sign of high effectiveness and productivity is bogus. This contradicts principles of one piece of work flow, queue size management, and capacity-utilization principles. It also conflicts with the concepts of collaboration and swarming that are strongly supported by cross-functional feature teams.
Some convincing studies about capacity utilization and its effects on productivity were done by Donald Reinertsen.7, 8
They are worth mentioning here because they can be tightly coupled with one of the Agile product development tools: Kanban.
Based on Reinertsen's Principle of Part-Time Resources (using part-time resources for high-variability tasks), maximizing the load of key resources with high-priority tasks is dangerous if more high-priority work is expected. Individuals who are highly loaded with high-priority work have low surge capacity, which is extra bandwidth that they can use against newly arrived high-priority work. What this means for Kanban teams that work on production support issues with different levels of severity is that the lack of surge capacity prevents a team from being able to switch to high-severity issues when they arise.
For example, imagine a Kanban team that has work of L1, L2, and L3 levels of severity moving through the same queue:
If a worker is always fully preoccupied with L3 work, his surge capacity very limited (literally, to his lunch hour), and this prevents him from picking up any additional L3 work, should such work enter a queue. This is particularly dangerous, especially if other workers that are fully preoccupied with L1 work (though having much higher surge capacity) have insufficient skill sets to handle incoming L3 work. It makes much more sense to optimize individual workload in ways that everyone, especially highly skilled specialists, have enough surge capacity (slack time away from L3 work) to be able to react to suddenly incoming high-priority work.
Challenges with Agile leadership
"Divorced from ethics, leadership is reduced to management and politics to mere technique."
-- James MacGregor Burns
Today, the most widely recognized Agile leadership role (above an individual team level) is an Agile coach
. Typically, Agile coaching is delivered by an external consultant who either engages with a company (client) directly and independently or
represents a specialized Agile coaching/training firm that deploys him on site.
Recently, companies began introducing internal Agile coaching practices by way of attracting external consulting talents and then converting them into full-time employees, and/or by retraining existing company employees to transform them into Agile coaches (native coaches).
In the former case, the following two questions usually come up:
Does engaging with a reputable Agile coaching consulting firm guarantee a top-notch Agile coach-consultant who will be deployed on site?
Once engaged, what are the odds that a consulting firm, intentionally or unintentionally, positions itself in such a way that its financial benefits from the engagement become more of its focus than value delivered to a client?
In the latter case, when internal coaches are used to help an organization with Agile transformation, the concerns are somewhere different:
Internal coaches that have not been exposed to the outside world (other industries, other corporate cultures) tend to have views that are narrowed to only what they have seen at their own companies (structure, culture) and, therefore, their ability to comparatively analyze organizational problems is significantly hindered.
Internal coaches, native or "naturalized" (defined further below), being full-time employees of an organization, are subject to evaluation, scrutiny, and performance measurements that are in conflict with what Agile culture needs. Such strict limitations, obviously, make it difficult even for the best coaches to provide (uncensored) reflection of a company.
When a company decides to procure an external coach and convert him into an employee (get him naturalized), a coach becomes subjected to the same type of evaluation and scrutiny as a native coach, or for that matter, as any other employee.
What does it mean for a role that historically is meant to be held by an independent, neutral third party?
An internal coach, unlike an external coach, cannot as freely reflect upon a company's ability to help itself by acknowledging its own problems and finding its own ways to resolve them. Now, a coach is part of
a company and the expectations of him are different. What an internal coach says about his own employer and the conclusions and recommendations a coach gives to his own employer will inevitably influence how an employer treats the coach. A coach's job is discovering/exposing organizational pain points, asking powerful, and at times uncomfortable, questions, as well as giving bold reorganizational recommendations -- a coach's ability to do so becomes a hostage to his or her fear of becoming the subject of criticism and reprisal. As coaches become (or remain) part of an organization, they are naturally forced to move away from being servant leaders to becoming more of personal achievers and politically correct commanders and controllers.
Image courtesy of https://www.goog le.com/search
Back to the external
coaching consulting model. One of the clearest representations of the Agile coaching dilemma has been described by Dan Mezick in his book The Culture Game
. The author vividly paints how important it is to define entry and exit criteria for every coaching engagement, as well as its duration, before
it begins in order to avoid excessive monetary transactions and long-lasting codependency between a client company and an external coach -- an indication of unethical coaching.
Today, unfortunately, most companies still rely on external Agile coaching expertise blindly, without questioning its effectiveness. Large-scale coaching engagements are frequently secured based on personal relationships, where SOWs get signed and monetary transactions take place, not
where transformation takes place and value gets added.
With external Agile coaching, when Agile transformations get done in one "big bang" and there is suddenly a high surge in Agile coaching demand by a client company, the Agile transformation company runs a fire drill and tries to immediately produce additional coaching staff, by turning to the marketplace and procuring independent coaches from the street and then reselling them to the client as "their own." This certainly does not guarantee to a client the good quality of a coaching resource, and it most certainly does not render such resource at a true net cost (the mark-up for these types of engagements is usually pretty high).
Another scenario is when a coaching role gets occupied by an individual with great coaching skills but very little practical, hands-on experience within Agile. Typically, such situations arise when a person comes from a completely different area of coaching (personal coaching, spiritual coaching, career coaching, life coaching, etc.) and effectively uses his "soft"/people facilitation skills to compensate for very superficial subject matter expertise.
Since Agile is primarily about product development, ideally the person who steps into the Agile coaching role should have some technical or at least semi-technical background. This would help him or her better understand and appreciate product development issues, and therefore provide better advice and earn a higher reputation.
Going by the same logic, when a company decides to build its own Agile practice, it is very important that selected individuals do a good share of observing and shadowing more experienced Agile coaches before taking their own initiative. Co-coaching with more experienced peers helps a novice coach not only capitalize on his understanding and practical know-how of Agile mechanics but also adopt proper behavioral patterns of being a servant leader and enabler, not a commander and controller -- something that is often seen when a coaching role gets filled by an internal person who was previously an authoritative figure.
Agile impact on individuals
Struggle for personal adaptation
"Today a thousand doors of enterprise are open to you, inviting you to useful work. To live at this time is an inestimable privilege, and a sacred obligation devolves upon you to make right use of your opportunities. Today is the day in which to attempt and achieve something worthwhile."
-- Grenville Kleiser
Agile affects professional careers and personal lives. So let's pause here for a moment and make an important distinction: Agile was initially introduced as a way to develop products
. Its purpose was not
to manage individual projects, as is incorrectly perceived by those who are familiar only with traditional software development. Nor was it a "plug-in" into any other method or framework that companies use. The purpose of Agile was (and remains) to get a product of the best quality to a client in the shortest time frame, at the lowest cost possible.
For many individuals, Agile is a great way to explore themselves, to reveal and further develop their individual potential. Agile favors innovative thinking, helps merge the gap between creative art and the science of technology, and assists in gaining the freedom of making choices and developing the Kaizen culture.
Since Agile originated primarily as the mechanism for product development, individuals with skills that are required for product development are able to adapt to Agile relatively quickly.
On the contrary, for those individuals whose skills are not directly related to product development processes, Agile adoption presents a significant challenge. Such individuals cannot effectively contribute to day-to-day activities needed for Lean product development, they don't easily fit into the flatter organizational structure required by Agile, and they cannot adapt to the absence of the command-and-control environment that prevailed under previous working conditions.
Image courtesy of https://www.actonmba.org
Agile is meant for true doers. If we recall Donald Reintersen's7,8
discussions about product development, the closer an individual is located to the automated production line, the more value he brings to the process. In terms of software development, this might be translated as follows: The closer an individual is to a code base, the more value he brings to product development -- and vice versa.
Let's take a look at a developer. In an Agile environment, a developer is given many more opportunities than in a non-Agile environment to explore his intellectual capacity, while thinking outside the box and coming up with innovative solutions. No longer does a developer obediently execute against frozen business requirements that were most likely put in silo by a business analyst, with minimal or no input from technology and with minimal or no exposure to real end users. In the latter case, by the time a developer starts coding, requirements are most likely stale and out of date. This ultimately leads to change requests and, most likely, to product changes when they are the most expensive to make.
In Agile, a developer has direct communication with a "buyer" (end-client or an empowered proxy, the product owner). A developer now has an opportunity to look at each business requirement (usually a user story) individually and offer a unique, at times innovative, technical solution with a very short feedback loop (response) from a client. In case of a positive feedback loop, a developer is encouraged and happy to claim a small credit for his win and a job well done. In case of a negative feedback loop, a developer has little damage control to do, as the amount of rework required is usually minimal.
What developers do find challenging, however, as they transition from a more conventional environment to Agile, is that they are not always able to work "on par" with other developers. In Agile (let's take the Scrum framework for example), developers on the same team might be at different levels of seniority and, even more undesirable, unevenly positioned with respect to each other in the same organizational hierarchical structure. This is a problem, as the lack of equality among team members prevents them from having effective collaboration.
But all in all, for a skilled developer who is willing to cross-train further and become a true T-shaped person (specialist in one field plus a generalist in one or more other fields), Agile is a land of great opportunities for personal and professional growth.
The situation is even more straightforward with QA. Let's make a note here about one very important precondition for scalable Agile: automated testing coverage.
If manual testing still predominates over automation, development will stall and plateau as soon as manual QA is not able to keep up with development. Ideally, test automation should begin with the first line of code or, even better, it should come before coding begins (e.g., TDD/ATDD). Success in Agile is not possible without test automation.
In Agile, QA involvement begins much sooner than in Waterfall, where QA gets to see a product only when development is practically done and when the discovery and repair of bugs is the most expensive. In Agile, QA's role is elevated
significantly, becoming a much more reputable role. The discouraging notion that we hear at times, "A QA person is someone who is not good enough to become a true developer," is no longer valid in Agile, as QA is now rightfully considered a member of the development team and not just someone who manually executes only by following a handwritten test case.
There is one big assumption, however: that a QA person is willing and able to become and/or remain technical. Any automation test tool requires some coding skills, and since true Agile cannot exist without test automation, the QA person should be comfortable with coding. There may be no need to become a full match to a senior programmer, but QA does need to come closer to a developer in terms of technical savviness.
This might present a challenge to those QA people who have done manual testing only. In Agile, manual testing is only temporarily valuable during initial sprints, when code base is limited and there are not too many features to test. Over time, as more code gets produced and more functionalities become available, manual end-to-end testing cannot keep up with development. Therefore, we need a machine.
Similar to developers, the ability to work "on par" with other team members may present a challenge. Here, the adjustment for QA people is more psychological and cultural than functional -- they have to be at the same level with other team members, regardless of their current position in the organizational tree.
Overall, if we assume that cultural adjustment does not present a serious challenge for a technically predisposed QA person, Agile also presents an array of opportunities in terms of gaining more hands-on knowledge, professional respect, and team recognition.
The role of the business analyst is clearly articulated on the www.iiba.org
website. According to the International Institute of Business Analysis, the role of the BA includes various types of analysis: systems, requirements, data, process, business intelligence.
Since the discussion of this paper focuses on Agile, and given the fact that the main intention of Agile is to improve product development practices, our main focus here is on BAs who participate in product development and serve the purpose of a primary conduit between business and technology.
Since in Agile product development the intent is to bring closer the business community (end-users) and technology (feature teams), if BAs want to stay close to the product development process and survive organizational flattening, they have to position themselves in one of the following ways:
Assume the role of product owner
Assume the role of product owner-proxy (if such a layer is justified)
Embed with a feature team as team BA
In order for a BA to be able to assume a product ownership role, his level of authority and executive decision-making power must be significantly increased. Today, in conventional settings, even very senior BAs cannot make final business decisions on their own, as they have to seek approval of more senior staff, including business stakeholders, sponsors, etc. Even in instances when BAs are able to formulate decisions based on the inputs of many, the cycle time from the moment the BA raises a question to the moment he is able to derive a conclusive decision and communicate it to IT is way too long to support an Agile development pace that is based on a short cycle times.
Looking at the situation realistically, it is highly unlikely that the BA will get empowered to a level that he could be rightfully considered the PO. For this to happen, most likely, the BA would have to jump a few hierarchical levels, something that does not happen frequently. Such BA empowerment is even less expected at large, enterprise-size companies than at small/mid-size companies, as reorganizational decisions take place much more quickly and a flatter structure is more natural in the latter. Therefore, although having a BA become a PO is possible, at most enterprise-level companies it is unlikely.
What is much more likely to happen is to have a BA assume the role of PO-proxy -- a role that is sometimes introduced to help the PO with his responsibilities. Introducing the PO-proxy role is much more common at large, enterprise-size companies than at smaller ones where a company's primary line of business is software development.
You may ask why. The answer is simple: A company that primarily generates its revenue from building and selling software to external clients cannot afford to have a multilayered product ownership structure. The risk of miscommunication and delayed response is just way too high. The PO role is taken much more seriously at smaller/mid-size software development companies. It is a full-time job and whoever takes it gives it his full focus.
At larger companies, however, having a BA or BA-like person assuming a PO-proxy role is much more common; larger companies are more tolerant of having PO-proxy roles. Meanwhile, the role of actual PO (head PO or chief PO) is given to a person with more stripes on his shoulders.
Here is a typical scenario:
The PO role is given to someone who is positioned high in the organizational food chain -- someone who is entitled to make final business decisions. But in the majority of cases, such a PO is more of a political figure who neither fully understands nor is being held accountable for performing his duties as PO. He is not so much a decision maker as a decision "signer," whereas decisions are recommended by someone else. This "someone" is typically a PO-proxy, a lower-ranking role that is almost immediately introduced after selecting the PO. The PO-proxy role becomes a buffer layer between the PO and real work.
Although there are instances in which the model of a single PO plus multiple PO-proxies makes sense (e.g., the PO is responsible for a product that is being built by multiple feature teams, where each PO-proxy supports each individual team), in general such additional organizational layers create unwanted risks: miscommunication, misunderstanding, and increased cycle time.
In his book Agile Product Development with Scrum
, Roman Pichler clearly describes how having BAs stepping into PO-proxy roles creates multiple problems that ultimately lead to "decrease of productivity and morale."2
Therefore, although becoming a PO-proxy is much more realistic than becoming the PO, it still does not provide a very effective solution for accommodating BAs, as the volume of BAs that each organization harbors today by far exceeds the number of available PO-proxy openings. This is a very basic supply-and-demand dilemma.
Let's face it, for a business analyst (or for someone at the same organizational level) to step into a PO's role is nothing but a great opportunity for career growth. It is a step up into the spotlight, gaining more visibility and authority, being presented with more opportunities to network and form useful professional relationships. Today, there are many BAs who are asked to do the heavy lifting for POs, including writing stories, backlog grooming, communicating with feature teams (especially offshore teams) -- pretty much everything except making final decisions. This creates a situation in which BAs do a lot of heavy lifting for little recognition.
Elevating BAs to PO-proxies (still much more realistic then becoming the PO), deputizing them to be not just backstage servants but rather front-stage leaders, creates a rewarding situation for them, whereby assuming a more important organizational position becomes very attractive.
Let's take a look at the opposite situation: An individual who already has a high-ranking job with a company is asked to step into the PO's shoes.
Such an individual already has a plenty of visibility, authority, and, most certainly, lots of day-to-day responsibilities. This individual's job has already been defined in "pre-Agile" terms with clearly formulated success/failure indicators, and, crucially, a compensation structure. Naturally, such an individual will not
genuinely embrace the additional PO role unless
the following conditions are met:
His other day-to-day responsibilities are minimized.
His compensation is increased to justify for additional efforts required to perform the second job.
There is a hybrid of the first two conditions: reasonable workload, reasonable compensation, no significant loss of organizational positioning.
In a majority of cases, companies attempt to fulfill the first condition but complete it only partially by simply doing an internal reorganization and appointing an individual for the PO role without
removing his existing responsibilities. There will always be some resistance along the way. This is why:
For a product manager or a key SME/business user to have more direct interaction with technology usually means performing "dirty" work.
For a product manager or a key SME/business user to step into a PO's role sometimes means stepping down in the organizational tree and giving up direct reports -- something that might be required to avoid conflicts of interest.
Regardless of overall workload for a newly baked PO, the type of work required by the role and how this work ties to organizational positioning and monetary rewards (e.g., potential bonuses) is usually the reason for low support.
Under the second condition (the likelihood of which, by the way, is not high at large organizations since salaries and bonuses are tied to organizational levels), yet another likelihood for failure is that no extra money would compensate for the extra time and energy that a worker needs to spend on doing a second full-time job. It is highly unlikely that an individual would be able to sustain twice the workload without having it affect his personal lifestyle. It is also unlikely that a company would be willing to increase an individual's compensation twofold to pay for a double effort. And as has been proven many times, "highly compensated heroics" never have long-lasting effects.
It seems that the only potentially workable option would be to create hybrid conditions under which, on one hand, the PO feels safe that his role within an organization did not depreciate (although the workload would decrease) and, on the other hand, he was able to give the required time and attention to Agile teams as needed. At the same time, the PO's compensation increase and rewards would have to be within reasonable terms to compensate for significant additional work, yet not cause serious exception to a compensation formula used by a company.
In his book The Art of Product Management
, Richard Mironov1
describes situations when the role of product owner in Scrum is fulfilled by a person whose day-to-day duties and responsibilities resemble that of a PO: product manager. The product manager is a conventional, outward-facing role of a product business owner. Of all potential candidates for the PO role in Scrum, the product manager seems to be the closest to the PO role.
Mironov goes into detail, outlining how the two roles (PO and product manager) differ, specifically stressing the fact that the speed of crashing/failure is much higher for a PO than it is for a product manager. This is due to the fact that things move much faster in Scrum and time frames to observe antipatterns and shortcomings are much narrower.
The most important thing that the two roles have in common, and the main reason for the lack of success: lack of engagement
One of the most questionable roles in Agile product development is that of the project manager.
Is this role still required? This topic is controversial and causes a lot of discomfort when raised. For many PMs, the uncertainty about how their jobs will be impacted by Agile is the reason why Agile meets resistance in the conventional PM world.
Agile adoption presents different challenges for technical managers and nontechnical managers. It is important to make a note of this distinction.
For mid-level technical managers, the dilemma is primarily psychological and behavioral. Technical managers are expected to have hands-on expertise and, if needed, should be able to roll up their sleeves and produce technical work relatively quickly. This is exactly what counts in Agile product development -- producing tangible technical work. The main challenge that technical managers face is their ability to let go of authoritative power and control of their subordinates. In Agile, tactical technical decisions are to be made at a team level, not at a technical management level. Psychologically, loss of such centralized control creates a problem -- it is discomforting. The situation may worsen if a technical manager needs to become a member of a feature team, where he starts working side by side with individuals who previously reported to him.
As mentioned in the discussion about POs, organizational flattening and the adoption of an Agile framework may translate into a loss of jurisdictional power for some, and technical managers are not an exception.
Nevertheless, this does not create a "potential job loss" situation for technical managers, as they are always able (or at least expected) to fall back on their primary technical skills and to integrate with feature teams. Alternatively, some individuals can get promoted from mid-level technical management to more strategic technical leadership, where they are not be so much involved in tactical work at the team level but are responsible for more strategic decisions and resource planning across multiple teams.
But again, just like the space for "upraising" BAs to PO-proxies is limited, so is the space for uprating mid-level technical managers into more senior positions: Supply and demand rules still apply.
It is important to note that the notion of senior technical leadership does not go away when Agile is introduced, as senior technical management is still needed. It is the abundance of mid-level technical management and the redundancy of their work.
Things are much more different for non-technical managers.
In Agile, let's take the Scrum framework as an example. The responsibilities of the project manager are evenly distributed between the PO and the team. The PO is now responsible for all strategic planning: product vision, product road map, time lines, budget, and scope (remains flexible most of the time). A team is responsible for all tactical activities: sprint planning, story estimation, task breakdown, work assignment and work flow management, various team ceremonies, etc. A ScrumMaster, selected by a team (the ideal case), usually picks up various team logistics, resolving inter- and intra-team impediments, brokering and facilitating ceremonies, negotiating with the PO, protecting a team from undesirable external influence, and escalating problems to executive management.
So is there anything left to do for a mid-level non-technical project manager in an organization that gets leaner and flatter as it undergoes Agile transformation, lightens its processes on all fronts, and gets rid of all its "procedural" waste? It all depends on how easily a non-technical PM is able to adjust.
Among other less apparent elements that are required for mid-level non-technical PMs to stay afloat in an Agile-transforming organization, the two main adjustment requirements are:
Mental shift away from command-and-control behavior
Ability and willingness to pick up additional technical skills that would make the PM more valuable in a product development process
The first is all about behavioral patterns. It is about accepting that a group of skilled professionals, when empowered, can make decisions about their own work better than any outsider who is not doing the actual work, especially if such an outsider is not qualified technically. Since Scrum (we continuously use this Agile framework as an example, as it is the most structured one of all Agile frameworks) implies self-direction and self-governance, any attempts to force anything upon a feature team will have adverse effects on both parties: on one hand, it will deteriorate Scrum and stall evolvement of a Kaizen culture, and on the other hand, it will marginalize a person who attempts to act as enforcer even further from where the real action takes place.
At this point, it is worth mentioning one very common anti-pattern that is often observed in large organizations as they undergo Agile transformation: The ScrumMaster role automatically gets filled by a former PM. Is this default assignment proper?
Automatic sanctioning of PMs with SM responsibilities may cause more harm than good, and this has proven to be the case on multiple occasions. Having PMs go out and get certified as ScrumMasters is not sufficient. If a PM mind-set remains, the person will never become a ScrumMaster. Unfortunately, mind shifting is not something that any certification can change. When the PM steps into the SM role but continues using his command-and-control tactics, it demoralizes the team and becomes its biggest impediment. The situation becomes even more dangerous if a newly baked SM who used to be a direct manager of other (one or more) team members now becomes their SM. Even if organizational reporting lines are removed, psychological dependency frequently remains and prevents team members from thinking and acting freely.
In his book Scaling Lean and Agile Development
(coauthored with Bas Vodde), Craig Larman explicitly warns about harvesting "fake ScrumMasters." Larman's quote, "Changing the title of someone to 'Scrum Master' while he acts like -- and is encouraged by the organization to act like -- a project manager"5
alludes to the fact that simply relabeling old roles into new ones does not cultivate better behaviors and does not improve culture.
Although there are many PMs who are willing and capable of undergoing a mind shift to become SMs, they still represent a fraction. Some PMs also still feel that becoming a SMs is a step down in their careers, a demotion in a way, because now they no longer have the power to control the actions of others. They feel discouraged by the situation and start seeking other career paths. Are their employers aware of that?
What is also worth mentioning is that, unlike the PO (we still refer to Scrum roles for the reasons mentioned above), the ScrumMaster position is rarely a full-time job. Unless the SM is sanctioned to support multiple teams, which is also not always desirable, it remains a part-time facilitator role. Under ideal conditions, the SM role can be picked up by any team member, or, what is even better, by a member of another feature team (this way, a team will be able to completely avoid a conflict of interest and ensure neutrality). Since the SM role is not a full-time job in the majority of cases, it also means that a person who holds it is expected to have specific functional (technical or semi-technical) responsibilities that can benefit a feature team in product development.
This brings the discussion to the following question:
Is a non-technical PM willing and able to gain additional technical skills and functional expertise to remain a valuable asset within an organization that is undergoing Agile transformation?
For many non-technical PMs, learning technology is not an easy task. It is not always easy to switch from many years of using conventional project management tools and techniques, such as a project plans, project charters, WBS, Gantt charts, and individual task assignment lists to Java console, class libraries, Web services API, automatic test tools, or Agile software development collaboration tools.
Often, project managers select less technical direction in their transition. They retrain as BAs and embed with teams, where they begin serving the purpose of PO or PO-proxy conduits, by supporting teams with business requirements (backlog items).
This shift certainly helps in retaining resources, which is always a positive characteristic of any corporate culture, but it only works if there is a very specific need for having a BA embedded with a team (this is more practical in distributed Scrum with offshore teams). Otherwise, creating an extra layer in the business-to-technology communication channel is not recommended. And again, supply-and-demand rules suggest that there are not enough team BA vacancies to accommodate all requalifying PMs.
Epidemic of certifications
"It is not enough to have knowledge, one must also apply it. It is not enough to have wishes, one must also accomplish."
-- Johann Wolfgang von Goethe
Over the last couple of years, the variety of Agile certifications has significantly increased. Specifically, entry-level certifications that attest to basic Agile knowledge (Scrum framework, specifically) are now available in abundance. The type and depth of knowledge that these certifications offer (usually superseded by a condensed training course) are similar and, to a large extent, they cover identical topics.
Image courtesy of https://www.google.com/search
If we search for reasons why there is such an abundance of basic-level Agile certifications, it will become apparent that it has become more about market share retention by certifying organizations that operate in the Agile arena than about delivering unique, universally acceptable attestation standards that, on one hand, truly reflect individual hands-on practical expertise and theoretical knowledge and, on the other hand, enable individuals to use earned Agile credentials for securing a competitive job.
This discussion is not about comparing one certification to another or suggesting which certification is better. It is also not about sharing research data about which certifications are more or less recognized by the industry. Most certainly, the intention is not to compare pricing or promote any particular certification -- they are all, when packaged skillfully, attractive. If a reader wants to fully grasp the variety of introductory Agile certifications available today, he can always do a Web search that will produce at least a half-dozen options.
The main purpose of this discussion is to stress the following point: There are a lot
of comparable certifications to pick from, and this creates unnecessary competition in a space where a universally recognized accreditation standard would be much more desirable.
Competition between organizations that takes the form of "my Agile is better then yours" creates confusion for those who want to get certified and capitalize on their fairly earned practical experience.
It also must be noted that by harvesting so many redundant certifications, the industry creates favorable conditions for "certifications collectors" -- individuals who obtain certifications for the sake of being certified. By doing so, such individuals skew the count accuracy of those who are really qualified for a job and fade the distinction between true practitioners and acronym collectors. At times, we see names of professionals in the Web space where certification abbreviations by far exceed the length of an individual's first and last names, combined. This is truly ironic. As practice shows, holders of multiple redundant certifications have little or even no practical hands-on experience in Agile space.
Finally, there is no statistical proof that would support a claim that any company-employer recognizes only
one type of certification over other types. This creates yet another challenge for professionals seeking employment, as they do not know which certification to pursue in order to increase their chances of being hired. If a company is looking for a certification abbreviation next to a name, instead of experience, it will most likely end up with an underqualified candidate who will further discredit a certification by not being able to live up to a company's expectations of his subject matter expertise.
Let's restate the initial purpose of this discussion. The goal was to make a reader come out of his comfort zone and think about issues that are often omitted from "happy path" Agile themes. The goal was neither to suggest any conclusiveness on the subject matter nor to steer the reader toward any particular actions.
After reviewing this discussion, each reader should be able to develop his own objective perception on a situation, his own independent view and perspective.
Still, how each individual perceives this information will, to a large extent, depend on such factors as:
Is the reader's perspective personal or organizational in nature?
How close to Agile transformation activities is the reader positioned today and how soon (if at all) will he be impacted by them?
Each factor by itself is influential, as are both of them in conjunction.
To some, this reading could be a great eye-opener and motivator to make personal adjustments to ensure job security and competitiveness in the job market. This could take on the form of becoming more valuable to one's own organization through pursuing education or training, or by becoming more selective of accreditations and certifications, or by planning an exit strategy to a workplace where conditions for non-Agile activities are favorable.
To others, especially to those whose views are more aligned with organizational ones (e.g., high-level executives, C-level officers), this writing could serve as a push toward internal adjustments: reorganization and/or retraining of resources, reconsidering employees&' awards and incentives models, revisiting contracts and SLAs with internal and external partners/vendors. By the same token, introducing inspection and validation checkpoints to ensure that there is continuous and gradual advancement toward long-term strategic goals would be another likely outcome of reading these pages.
Further, there will be a certain percentage of readers who will downplay the significance of this writing because of confidence in their "safety zone" away from Agile transformation (e.g., moved to a different department or company) or because they just do not expect a full impact of Agile any time soon (e.g., due to its slow adoption by the organization they work for).
Lastly, there will be parties (varying from individual employees to service providers to client companies) who will react to this discussion with disapproval and defensiveness for self-serving reasons. Most likely, such a reaction would be proportional to their level of involvement with Agile, from the perspective of business development, commerce, and enrichment. Here, fears of becoming a subject of scrutiny and an "ethical audit," should some of the issues that are being raised here find their way via wider broadcasting channels, will be the main driving force.
Endnotes and references
Mironov, Rich. The Art of Product Management: Lessons from a Silicon Valley Innovator. Createspace, 2008.
Pichler, Roman. Agile Product Management with Scrum: Creating Products That Customers Love. Addison-Wesley Professional, 2010.
Poppendieck, Mary; Poppendieck, Tom. Leading Lean Software Development: Results Are Not the Point. Addison-Wesley Professional, 2009.
Cockburn, Alistair. Agile Software Development: The Cooperative Game (second edition). Addison-Wesley Professional, 2006.
Larman, Craig; Vodde, Bas. Scaling Lean and Agile Development: Thinking and Organizational Tools for Large-Scale Scrum. Addison-Wesley Professional, 2008.
Logan, Dave; King, John; Fischer-Wright, Halee. Tribal Leadership. HarperBusiness; reprint edition, 2011.
Reinertsen, Donald G. The Principles of Product Development Flow: Second-Generation Lean Product Development. Celeritas Publishing, 2009.
Smith, Preston G.; Reinertsen, Donald G. Developing Products in Half the Time: New Rules, New Tools (second edition). Wiley, 1997.
Mezick, Daniel. The Culture Game: Tools for the Agile Manager. FreeStanding Press, 2012.