In a competitive economic environment, IT organizations must get higher-quality software into the hands of their users and customers faster than ever before, as they react to business demands and change more rapidly. Turning good ideas into marketable features quickly is a business imperative for every enterprise. Getting features "out there" faster and at high levels of quality is the first step. The next step is to rapidly collect feedback from users to guide the next set of ideas.
First, deliver features better. Then, deliver better features better!
In a traditional organization, communication among business, development, QA, and operations occurs on many different levels and often leads to confusion. Handoffs among the different groups involved in the overall process introduce long wait times and disruption. Slow, error-prone manual processes, delays, and lengthy fix cycles inevitably lead to infrequent releases. And when the organization infrequently releases large numbers of changes and fixes in one go, the go-live is stressful and often followed by days or weeks of post-release emergency patches.
is a set of processes and practices that radically removes waste from the software production process. It enables faster delivery of high-quality functionality and sets up a rapid and effective feedback loop between the business and its users. As a result, an organization delivers better software and has more satisfied customers. Continuous delivery helps reduce the time to market from weeks and months to days or hours!
Challenges of the traditional approach
The following are challenges that arise in each step of the traditional approach.
- Mismanaging requirements: The business analyst tries to specify up front the features as completely as the business can imagine them. Inevitably, this involves second-guessing the details of what customers actually want, leading to many features that do not quite meeting their expectations. From the perspective of delivering business value, there is already a built-in time-to-market delay, because sufficient features need to be gathered before work on a single feature can even begin.
- Coding in silos: Developers work in isolation, each making changes to their copy of the application code without being aware of any possible overlapping changes made by their colleagues.
- Late code integration: After weeks of working on an isolated copy of the code, the new features now need to be combined or "merged" into one single codebase. Late and infrequent integration is a typical recipe for missing deadlines and introducing significant risk.
- Manually deploying code: Manual deployment that requires the development team to hand off code to operations is error-prone.
- Testing — lowering the bar after manual QA: The lengthy process of executing the test plan gets underway. With many important features threatening to be delayed by failing tests and mounting pressure from the business, it is decided that only a few of the defects are indeed showstoppers that must be fixed before continuing. A patch is quickly rushed out, only the few scenarios are verified, and the updated version is passed by the QA team. The release manager now creates a support ticket to release the patched version to production, accepting the fact that the remaining requirements, originally laid out, failed the test. These will need to be addressed in an urgent maintenance release.
- Preparing the release — the preproduction bottleneck: Before the new version can be released to production, it must pass a final set of tests in the preproduction environment. This is required to increase the confidence level that the go-live will succeed, since the acceptance environment differs substantially from production.
- Going live — the quarterly release cycle: The new version of the application has been approved just in time for the next available quarterly release slot. Operations manually deploy new application versions, which are time-consuming. After deployment, the QA department takes more time to verify the releases, and then the application is live.
From a business perspective, it has taken months
to get only a small subset of features.
The principles of continuous delivery
One of the appealing aspects of continuous delivery is that it gives the business much more direct control of feature development from "throwing a bunch of the requirements over the wall to IT and waiting months to see what comes out" to "having an idea today and being able to follow all the steps to go live tomorrow."
The Value Stream Map for the software delivery process is a proven tool for identifying and progressively eliminating waste in a process. The goal of continuous delivery is to create a constant flow of changes to production by an automated software production line called the continuous delivery pipeline
. The pipeline breaks the software delivery process down into a number of stages. Each stage is aimed at verifying the quality of new features from a different angle to prevent errors from affecting the users.
The pipeline provides feedback to the team and visibility into the flow of changes to everyone involved in feature delivery. This pipeline can be broken down into the following stages:
- Build automation and continuous integration
- Test automation
- Deployment automation
The standard approach to testing is to first develop new features and then test them. As a result, testing becomes a roadblock to going live. Inevitable delays cause testing to be cut short, resulting in the low quality of many released features.
Developers implement new features that are integrated into the central codebase on a continuous basis, built, and then unit tested. Before a developer implements a feature, they write a failing unit test for the piece of code about to be written (test-driven development
), and the change is added to the same codebase being worked on by all other developers on the team. The updated codebase is now verified again by the continuous integration server to check that the new code does not conflict with changes made by other developers. This is the most direct feedback cycle that tells the development team about the "health" of their application code.
The integration build verifies that the feature not only passes the automated acceptance test for a specification (regression testing
) but also meets system requirements for security, performance, and availability (nonfunctional requirements
). The speed and reliability of automatically provisioning and configuring the production-like test environments allow accelerating throughput by performing many types of tests in parallel, ensuring that the high-quality level verified in the pipeline will carry over to production. Most important, the status of each feature in the pipeline is immediately visible, and progress is highly predictable at all times. Everyone involved in feature delivery — from business and development to QA and operations — has complete visibility into every step of the process.
If a new feature does not cause any failures in the pipeline, it goes to the next fully automated production deployment. Since the preceding stages have verified the overall quality of the system, this is now a low-risk step. The deployment can be staged, with the new version being initially released to a subset of the production environment and monitored before being completely rolled out. The deployment is also automated, allowing for reliable delivery of new functionality to users within minutes whenever this is needed. Using additional deployment strategies, such as canary releases and dark launches, an incremental deployment strategy is applied that minimizes the risk of downtime and allows the delivery team to discover how the application behaves in a production environment without immediately impacting all users. The insight into real-life customer behavior enables the organization to learn faster and make better business decisions, day after day.
Because deployments are carried out frequently as an application moves through the delivery pipeline, the deployment process itself is tested every time a code change triggers a new pipeline run. Problems in the deployment configuration are quickly highlighted and can be fixed in the same manner and at the same time as problems in the application code.