If you don't get the user needs right, it doesn't matter how well you execute the rest of the project. Requirements are the foundation for all the project work that follows. The purpose of a software development project is to build a product that provides value to a particular set of customers. Thus customer involvement is the most critical contributor in eliciting requirements.
Let us go through a famous survey on this topic. The Standish Group surveyed IT executive managers in 1995 for their opinions about why projects succeed. The three major reasons that a project will succeed are user involvement (15.9%), executive management support (13.9%), and a clear statement of requirements (13%). Opinions about why projects are impaired and ultimately canceled included incomplete requirements (13.1%) and lack of user involvement (12.4%) at the top of the list. In his now classic ROI presentation at XP 2002, Jim Johnson shared research regarding the productivity achieved by those software projects that do deliver something. Among these, he found that 45% of product features are never used, 19% are rarely used, and 36% are routinely used. This tells us that traditional requirement engineering practices are not helping us deliver value to our customer through our traditional practices.
In my experience, traditional requirement engineering practices applied through Waterfall models have numerous limitations, including the following:
What happens in Agile?
Why? It is difficult to correlate each requirement with the corresponding end business value it maps to. Thus it is becoming difficult for the development team to understand the business reason behind each of the specified requirements. This also leads to ineffective prioritization and thus impacts ROI-based decisions.
Who? It is difficult to understand the mapping between each requirement and the person who uses it. It creates a huge gap between users and the developers.
What? It is difficult to keep requirement documents current and synchronized, especially when the project involves multi-site and multi-group involvement.
Users’ perspectives or business analyst’s interpretations: Often, we see standard requirement specifications leaning toward the business analyst's interpretations rather than toward clear capturing of user perspectives.
De-focus: Lengthy requirements descriptions may derail developers trying to understand the exact intent behind the requirement specification.
Changes happen: It is inevitable that requirements will change. Business needs evolve, new users or markets are identified, business rules and government regulations are revised, and operating systems change over time. When changes happen to overall business goals, it is difficult to reflect those changes across the associated requirement specification documents pertaining to various products.
Exact user acceptance criteria are often not clearly known to developers when they build the features. This often leads defective products.
Even the best requirements documents cannot -- and should not -- replace human dialogue. We often see limited interactions with users and more focus toward documents and processes. This leads to defective product feature implementation.
Validation of requirements does not often take place against business ROI and user satisfaction. This leads to less usage of product features and less profit.
Periodic feedback is not enabled between users and developers in the Waterfall development model till user acceptance testing. Thus it incurs huge costs to incorporate user feedback as a result of user acceptance testing.
Kent Beck coined the term "user stories" in Extreme Programming in 1999. A user story is a short, simple description of a feature told from the perspective of the person who desires the new capability, usually a user or customer of the system. The essence of user-centric and usage requirements elicitation is to focus on what the user wants to have, not what the user wants the system to do.
Agile projects, especially Scrum ones, use a product backlog, which is a prioritized list of the user stories to be developed in a product or service. Let us learn these basics in detail.
The user story is accompanied by acceptance criteria
A user story typically follows a simple template:
As a <type of user>,
I want <to perform some task>
so that I can <achieve some goal/benefit/value>.
User stories need to be confirmed with the 3C guidelines below, suggested by Ron Jeffries:
Card: User stories are traditionally written on index cards or sticky notes, in short form. User narratives further explain these cards. Thus the main intention is to describe the user story in short form to allow common understanding of the user need among all stakeholders.
Conversation: User stories shift the focus from writing about features to discussing them. In fact, these discussions are more important than whatever text is written.
Confirmation: Acceptance tests confirm that the story was delivered correctly.
Let us take an example.
User story title:
Customer withdraws cash.
I want to
withdraw cash from an ATM
I don't have to wait in line at the bank.
Acceptance Criterion 1:
that the account is creditworthy
And the card is valid
And the dispenser contains cash,
the customer requests the cash
ensure the account is debited
And ensure cash is dispensed
And ensure the card is returned.
Acceptance Criterion 2:
that the account is overdrawn
And the card is valid,
the customer requests the cash
ensure the rejection message is displayed
And ensure cash is not dispensed.
Product backlogs and epics
User stories are written throughout the Agile project. Usually a story-writing workshop is held near the beginning. Everyone on the team participates, with the goal of creating a product backlog that fully describes the functionality to be added over the course of the project or a three- to six-month release cycle within it.
Some of these Agile user stories will undoubtedly be epics. Epics are large user stories and will later be decomposed into smaller stories that fit more readily into a single iteration. Additionally, new stories can be written and added to the product backlog at any time and by anyone.
While a product backlog can be thought of as a replacement for the requirements document of a traditional project, it is important to remember that the written part of an Agile user story ("As a user, I want . . . ") is incomplete until the discussions about that story occur.
Anyone can write user stories. It's the product owner's responsibility to make sure a product backlog of Agile user stories exists, but that doesn't mean that the product owner is the one who writes them. Over the course of a good Agile project, you should expect to have user story examples written by each team member. Also, note that who writes a user story is far less important than who is involved in the discussion of it.
Agile helps us get the requirement right
Why? The user story clearly describes the goal or benefit the user is trying to achieve through it. Thus it is easy to correlate each user story with the corresponding end business value it maps to. Using KANO and MoSCoW prioritization techniques, the product owner can optimize the product's ROI.
Who? The user story starts with the persona who is needed this functionality. Thus it connects users who consume the capability with the developers.
What? In Agile, one product will have one product backlog. The product backlog can be subdivided into product area backlogs for the respective multiple groups. In this format, user stories can be grouped under epics so that we can establish a common understanding about the user cases across the product backlog.
Users' perspectives: User stories are written to capture user perspectives only. We also capture user acceptance criteria up front for the related user story. This indeed offers great help to the development team.
Focused: User stories are short forms of the user’s needs. We capture all the related information in user narratives. This helps the development team understand the exact intent of the user need.
Changes happen: User stories recognize the importance of change. User stories absorb change through the means below.
If the scope of the user story becomes large, they can be split into smaller user stories
By adding "conditions" to acceptance criterion
Exact user acceptance criteria are clearly known to developers up front. This leads to better connection between users and developers and thus facilitates a collaborative, value-driven path.
User stories using the 3C guidelines are driven by human dialogue. Please remember that the written part of an Agile user story ("As a user, I want . . . ") is incomplete until the discussions about that story occur.
Validation of user needs is clearly documented through acceptance criteria, and the same is captured for each user story. This ultimately leads to user satisfaction, as it is communicated early to the development community.
Periodic feedback is enabled through sprint reviews in the Scrum framework. Thus, users are given working software well up front and their feedback is periodically incorporated into the product iteratively.
Agile projects are successful three times more often than non-Agile projects, according to the 2011 CHAOS report from the Standish Group. The report goes so far as to say, "The Agile process is the universal remedy for software development project failure. Software applications developed through the Agile process have three times the success rate of the traditional Waterfall method and a much lower percentage of time and cost overruns" (page 25). In this process, Agile user stories certainly act as building blocks for software project development success.
Cohn, Mike. User Stories Applied: For Agile Software Development. Addison-Wesley Professional. 2004.
Wiegers, Karl E. Software Requirements. Microsoft Press. 2003.