Waterfall methodology has been around for so long that the software organizations that practice it — and their customers — are completely in tune with the process. Recently our resident Agile guru pointed this out when commenting that, with Agile, customers or product owners can't give engineering a memory dump. In other words, they can't give engineering a dump of requirements at the start, because Agile mandates constant collaboration, in which requirements are aligned with the team and the business as well.
This got me thinking: Internally, within a product-development organization that uses Agile, by hook or by crook the product owners have to fall in line. They'll learn the whole concept of a product backlog and backlog grooming; they'll take up the tasks that Agile expects of them. However, in a project-driven organization where customers are the custodians of the requirement, are those customers ready for Agile?
I recall many times during my consulting stints when three to four months were dedicated to "discovering" customer processes, drawing requirements, getting a requirement document sign-off — all before we could kick off the project. The calculations of the entire resource allocation, of commercials and timelines, were based on these "requirements." This is what the IT community taught customers to expect from a project.
Today, as the IT community is adopting and adapting to Agile practices, are we doing enough to educate and inform our ecosystem — our customers?
What follows are my views on some of the processes or engagements that customers need to adapt to in the world of Agile.
Memory dump versus iterative requirement grooming
In Waterfall methodology, requirements kicked off projects, with customers stating them and consultants documenting them. The requirements were never again revisited with the customer, since the customer had already signed off on them. Projects would begin only after requirement gathering was over.
In Agile, however, requirements can, at the very start, be coined as "epics" and regularly need to be broken down into stories. At each sprint planning, all stakeholders (including the customer) agree on stories (otherwise known as features and requirements) that need to be developed and tested in the upcoming sprint. This should be leveraged as a "wow" factor by the project team, to help customers understand that they can be completely in tune with what's being developed, and they can also prioritize features based on their business needs.
However, there are challenges:
- Participation: Customers have to sign up for the sprint demo and sprint planning. If skipping these is the norm, and product owners regularly become proxies for the customer, the benefits of Agile can never be experienced or implemented.
- Alignment of teams: The functional, development, and QA teams have to break out of their Waterfall mindsets and truly work in tandem in every sprint, aiming at successful sprint demos without sweeping issues under the carpet.
Management estimations versus Scrum team story-point estimations
In Waterfall, project timelines are drawn out from the word go. A few weeks of requirement gathering are expected to enable the project and account managers to draw a perfect project plan, with resource allocation, user acceptance testing, and rollout dates all drawn out. A project manager worth his salt will always build in a 20 to 30 percent buffer.
Agile turns this process on its head. It entails that the Scrum team (not management) should story-point the stories and sign up for consuming those story points based on the team's velocity.
The challenges here are:
- Project planning: I believe a timeline is the biggest challenge that projects face. The idea of always having a project end-date (which, as per various studies, aren't met 80 percent of time in Waterfall) is so inbred in IT organizations and customers that reaching the possible project end-date based on story points is going to take a lot of unlearning before it can happen.
- Ongoing estimations: Product owners, along with customers, need to participate in the Sprint planning to help the team plan, scrub, and story-point the product (requirements) backlog. Then they can see what can be accomplished in the coming sprint and what the spillovers are from previous sprint.
- Team velocity: The Agile methodology entails that the team be mature enough to know its velocity, experienced enough to size the requirements, and empowered enough to commit to the timelines.
End of development versus end of sprint demo
Sprint demos require the Scrum team to have a working, demo-worthy piece of code, but they also require commitment from the key stakeholder: the customer.
The challenges to this approach:
- Constant review: If the engineering and QA need to adapt to three- to four-week cycles to create a demo-worthy piece of code, the customer and product owner also need to adapt to the constant review process. The success or failure of a story eventually leads to how well the Scrum team understood the story, thereby calling for a review of the story itself. Gone are the days of "But I thought you understood what I told you then." If the sprint planning and even the daily stand-ups led to a communication gap, the entire team needs to revisit a lot of ground, requirements just being part of it.
Time and money projects versus fixed cost
The Waterfall method enables IT organizations to negotiate contracts as fixed-cost bids or time- and material-based bids. The bid type is driven by competitors' bids, the possible length of the project, and the budget and negotiating strength of the customer.
However, in the Agile world, wouldn't it make sense for everyone to have T&M contracts? Here's why:
- Iterative, sprint-driven practices give all stakeholders the capacity to stop the project after any sprint. (Assumption: At the end of any sprint, the team has a working code.)
- Sprint planning enables all stakeholders to access and analyze what can be achieved in future sprints based on team velocity.
- Product backlog grooming enables requirements to be aligned to business needs before every sprint, which enables customers to prioritize based on business goals. For example, enabling integration with PayPal for online shopping from the website can be prioritized over a partner dashboard, thereby achieving an online presence before the marketing campaign "Buy Online" goes viral.
These are few of the aspects of Agile that I believe IT organizations must adapt to. But an important point is that customers must also change the way they engage in the Agile world.