"The best architectures, requirements, and designs emerge from self-organizing teams."
-- Principle #11, Agile Manifesto
Emerge: To come into being through evolution
To rise from, come out into view
-- Merriam-Webster Online Dictionary
I spent the better part of a recent Sunday afternoon (when I should have been watching golf) in my backyard, pulling weeds out of my gardens. How does this happen, I wondered? How do weeds suddenly fill my gardens to the point that I have to spend an entire afternoon pulling them out? After all, I just weeded these same gardens a month ago, and now the weeds are bigger and more numerous than ever!
A similar situation happens with the children we know, who one day fit in the palms of our hands (almost literally) and are off to their first day of school the next. I often run into people who haven’t seen me since I was a child and they always state, much to my annoyance, "I remember when you were this big. . . ." while holding out their hand flat, palm down and close to the ground, indicating that I used to be smaller than I am now. No kidding!
Why are we so amazed at the growth weeds experience in a month, or children experience in a few years? Our amazement is due to the seemingly instantaneous nature of this growth from one time period to the next. We would be more accurate in our thinking to say that weeds emerge
over the course of a summer month or that a four-year-old child attending her first day of school emerged
during recent years. We need to think about Agile architecture in much the same way . . . it emerges
After switching to an Agile approach to software development, we often struggle to find a place in the process for architectural planning and implementation. We became used to planning all of a project's architectural needs up front and now worry that focus will be lost with an Agile approach. While it's true that designing, building, and maintaining a robust architecture doesn't come for free in Agile, architectural excellence is still within reach.
Agile teams still complete architectural work, but it's done differently than before. Instead of a big up-front design where decisions are made about the architectural needs for an entire project, Agile teams take an incremental and deliberate approach. That is, we can't just expect it to happen naturally and then wake up one day down the road with a product that no longer meets our clients' needs. We need to maintain focus on architecture throughout the entire process.
This is not easy to do, however. In his book Succeeding with Agile: Software Development Using Scrum
, Mike Cohn acknowledges the difficulties with emergent design on Agile projects, noting a few specific "realities" that Agile teams experience: Planning work and dividing work up among team members is harder, team members can feel uncomfortable not having a design first, and rework becomes inevitable. Fostering an environment of "intentional yet emergent" architecture is not easy. We need to accept these realities in order to determine the best course of action in our specific environment. We must master incremental planning despite its challenges. We must embrace the uncertainty of not having it all figured out at the start of a project. And we must understand the cost and benefit of the constant rework that comes along with incremental design.
It's important to keep in mind that technical practices are the key to achieving technical excellence. Proper unit and automated testing, continuous integration (CI) environment that builds at least daily and executes the unit and automated test suites as part of each run, pair programming (as appropriate), refactoring, and collective code ownership are all extremely important. This topic is explored further here
Now that we're all on the same page with regard to the challenges involved with an emergent architecture and the requisite technical practices, let's take a look at a few principles and strategies that Agile teams can leverage.
Principle: Think big, act small, fail fast, learn rapidly
This principle is taken from the Poppendiecks' book Lean Software Development: An Agile Toolkit
, and it represents a good starting point for our discussion. Many, if not all, of the principles and strategies discussed below fit into one of the phrases in this principle.
"Think big" implores teams to maintain an overall vision and direction for the architecture of the system. An architecture that is incremental yet emergent doesn't mean that no one is anticipating and looking toward the future. Someone needs to be plotting out the strategic architectural direction of the product and anticipating the needs of the business. This is not a big up-front design, however; it can take the form of an architectural vision document, an architecture team that supports the Scrum teams within an organization, or the Scrum team members being responsible for the architectural vision. The items that are prioritized as a result of the architectural vision end up on the architectural runway, but more on that later.
"Act small" is the incremental part of Agile architecture. Teams need to keep the big-picture architectural vision in mind while delivering small pieces of valuable software to the business. Those small "increments" must be of high quality and include consideration and possible incremental implementation of the architectural vision. Pieces of the emerging architecture should be built as part of a user story that is driven by business need and delivers value to the end user. This is often decided during grooming sessions that include the team and the product owner.
Mitigating risk is another topic addressed by this lean principle. "Failing fast" allows a team to front-load risk in its projects and, by doing so, deal with risk early and take the necessary steps to alleviate its impact on the project. This is complemented by the idea of "learning rapidly," as it's of no use to fail fast if teams do not learn from their failures. Teams need to identify and overcome challenges as early as possible to ensure the success of their projects.
Principle: The last responsible moment
This is one of my favorite principles and a general truth of life. By delaying decisions to the last responsible moment, we have the best and most current information at our disposal with which to make the best decision possible. The last responsible moment is defined by the Poppendiecks as "the moment at which failing to make a decision eliminates an important alternative." Boehm proved with his Cone of Uncertainty
that decisions made early in a project are often wrong, as they are made with limited information containing many assumptions.
Principle: Do the simplest thing that could possibly work
Since by their very nature emerging architectures are incremental, they also need to be simple. The Agile Manifesto suggests that simplicity is "the art of maximizing the amount of work not done." Teams should focus on delivering the simplest thing that supports the product and the needs of its clients while staying away from gold-plating, or "shaving a yak
Strategy: Sprint Zero
Sprint Zero is something that occurs at the start of a project, release, or formation of a new team. It allows for team norms to be established, required infrastructure to be put into place, and overall release preparation to be done. This overall release preparation can include user story grooming and estimation, filling out the team's backlog and architectural runway, reviewing the product and architectural vision, and planning for the first sprint. Sprint Zero should be the same time-boxed length as all other sprints and should occur only at the start of a project, release, or formation of a new team.
Strategy: Architectural runway
An architectural runway is a backlog maintaining the priorities of the product from an architectural point of view. The runway will contain priorities that reflect the "-ilities" of a product, such as scalability, maintainability, usability, reliability, extensibility, security, reusability, etc. An overall architectural vision helps focus the runway, while the runway helps the team focus on constantly building capability into the product. It's described as a runway because (unlike the product backlog), if the runway isn't long enough, the plane crashes. Paying an appropriate amount of attention to the architectural runway will prevent the product from crashing and enable it to scale with the challenges of tomorrow. Dean Leffingwell is a proponent of the architectural runway in his SAFe (Scaled Agile Framework
A spike is a concept taken from Extreme Programming and refers to a time-box that is used to complete research or design or otherwise mitigate risk or prove a concept important to the project. A spike doesn't have a business deliverable, and it is focused on providing an answer the team needs in order to move forward. Spikes can range from a few hours to a few days and should not be any longer than a sprint. The results of the spike can be discussed in the daily Scrum, sprint planning, or the sprint review meeting as appropriate.
Principle: Prove it with working software
Scott Ambler put it this way: "[P]rove it with code
." Any architecture that the team thinks it needs or any decisions that the team needs to make about the product's architecture should be proven with code. This ensures that teams aren't just designing concepts but are designing and implementing architectural necessities that work. The Agile Manifesto espouses this idea in the principle that "working software is the primary measure of progress."
Principle: Trust the team to do it
Last, we need to trust the team to "get 'er done!" Agile teams are cross-functional and made up of highly trained, highly skilled, highly thoughtful, and highly creative professionals. Assuming the team has all of the members required to reach the end product and exists in an environment supportive of Agile principles, there is every reason to believe that the team will be successful.
I started this article discussing weeds and children, whose growth is an example of emergence, and ended up using that as a jumping point into principles and strategies that can be used to ensure the success of an intentional yet emergent architecture.
Apparently articles, like successful Agile architectures, emerge