Many organizations have joined the race to become Agile. And in the competition to be Agile as soon as possible, many of these organizations have converted their Waterfall model to multiple quarterly mini Waterfalls. A mini Waterfall not only fails as a long-term solution but it's also unable to deliver quality because of resulting bottlenecks in testing.
Below is a typical Waterfall process flow based on a high number of features in a release:
- Suppose in the past the company delivered 12 features in a 12-month Waterfall. Based on this output, upper management now wants three features per quarter.
- The issue with delivering three features per quarter is that some features are small whereas others are broad or complex. In the case of delivering fewer features with varying complexity per release, a feature can include many INVEST (independent, negotiable, valuable, estimable, small, testable) stories.
- When trying to deliver huge features in a quarter, things run late and testing is a crunch. In a worst-case scenario, the delivery date is pushed a few days back, which is also against the practice of a sprint timebox.
- As the result of a crunched test cycle, quality is degraded or the next quarter's automation backlog includes spillover.
When circumstances outlined in 1 to 4 are repeated over a few mini Waterfalls, the amount of regression coverage increases, and testing becomes bottlenecked in each release.
The solution to a problem like this has multiple parts.
Convince higher management and sales to produce smaller stories and implement a user feedback loop. This also includes more transparent thinking, whereby the entire development team has a chance to interact with the client.
Ensure that middle-level management and the team have an understanding of a cross-functional team in the sense that the developer and tester roles are interchangeable.
Evolve from a tight team with work handoffs to a cross-functional team.
For example, if you have a team of:
- 5 quality assurance engineers
- 5 front-end developers
- 5 back-end developers
Your new team can be:
- 4 quality assurance engineers (QAE)
- 3 software development engineers in test (SDET)
- 4 front-end developers
- 4 back-end developers
We took one member from each team and created an SDET team. Keep in mind that the team of SDETs can be a rotating team.
The key difference between QAE and SDET is that QAEs work more toward test plans, test-case generation, and test execution for the current release. In contrast, SDETs work toward a test automation framework and automation of test-case execution. Hence, your team of QAEs should have broader background and expertise in testing, whereas SDETs should have a broader background in the programming platform that you are on.
The advantage gained from the SDET team is that it will allow your team to avoid a testing bottleneck. After you have people rotate to an SDET team, the team will develop the mindset of a cross-functional team.
Along with the above transformation of people, drive another transformation of the testing strategy.
Traditionally, companies that have been following the Waterfall model tend to have less coverage of unit tests, an average number of integration tests, and in-depth end-to-end tests of all scenarios.
In this model, a lot of time is spent on in-depth end-to-end tests. To understand this model, we can use a light switch example. Suppose you have eight light switches. Since a light can have two states, on or off, we are looking at 256 end-to-end combinations (or 2^8=256 ) for end-to-end test cases.
Many Agile organizations are following a reverse combination, which allows them to ship in a shorter amount of time.
- In-depth unit/component tests
- Average number of integration tests
- Only "primary" coverage of end-to-end tests as part of the Definition of Done (DoD)
Out of the 256 combinations, we decide that users are looking at 50–70 combinations in their daily use. All others are rare combinations, and stakeholders do not care about those combinations that much. The remaining combinations can be considered secondary end-to-end test cases.
Based on your DoD, you would commit to end-to-end coverage of 50–70 end-to-end test cases. The remaining end-to-end test cases should be tested, but they would not impact your release. If you are low on resources, then those cases can be used as a test pool for ad hoc or exploratory testing.
When you have an SDET team and focus more on unit testing, you can automate accordingly.
For example, if you have a Web-based product with REST, then unit and component tests will be automated using REST automation buckets and 50–70 end-to-end tests will be automated using UI automation buckets.
Over time, the testing side will have fewer QA members and more SDET members. The team of SDETs will be automating some part during a new feature release and finalizing automation in the sprint following the current sprint release.
After completing the above steps in multiple sprints of the SDET team work, and when the regression suite is robust and reliable, then your organization can start thinking about how to move toward monthly, weekly, or daily releases, which is an ultimate goal many companies aim for.
If you want to have a successful Waterfall-to-Agile migration, then consider testing in parallel with changes to the development cycle.