Large IT organizations with several business initiatives and multiple projects being executed in parallel require agility and collaboration at two levels: the project level, which is always where it starts; and the application level, which is often overlooked or at least not well defined. Let me explain.
In one of my previous engagements, I got an opportunity to work for a few years with product teams that were very keen on transitioning to an Agile model from a rather rigid and traditional method. We introduced several Agile practices and learned a lot about what works and what needs to change in an enterprise. What we practiced, though, was not Scrum in its true form. However, it was certainly far more collaborative and visible than what we started with. This proposed model is based on lessons learned from things we did and things we should have done! If there are already practices in place that address this, I am hoping to hear about them.
Figure 1: Typical project flow in an enterprise
Figure 1 depicts the organization and project flow that I have in mind. The business groups, basing decisions on customer and market needs, come up with requirements that they would like implemented. Each business group works with one or more product managers to finalize the functional requirements. The SMEs from the engineering group often participate in detailing out the functional requirements. A project manager helps identify the various stakeholders, size the effort, draft a plan, and seek the necessary funding approval. This then leads to a detailed project plan and the execution of the project. This process, from articulating the business requirements to defining the functionality and finally developing and implementing, often takes several months (or sometimes years) of a traditional Waterfall development cycle.
Figure 2: Project-level Scrum
Waterfall-to-Agile transformation exercises are centered on the project. This is the obvious place to start with Scrum, as it is here that business, product, developers, and quality control teams need to actively participate and collaborate. Budgets are assigned to projects, and management closely monitors the issues and status of each project. Essentially, the focus of the organization is on project execution. Scrum backlogs, stand-up meetings, and burn-down charts provide the necessary visibility required at the project level.
Sprints and teams
A simplistic scenario is when a project requires changes to just a single product, application, or service. In a large organization, that is seldom the case. A project would typically impact multiple engineering applications (or products or services). Scrum practitioners address this by defining multiple teams, each with a different focus. Sprint backlogs are categorized into different application backlogs so the team assigned to it can work on it.
The bigger challenge that Project Scrum does not address is one that the developers face at the application level. Multiple teams, each working in the context of a different project, are likely to be working on the same application at any time. To manage the changes to the application effectively, multiple code branches are created, one for each team. This makes it easier to treat each project as independent from the others and prevents teams from stepping on each other's code during development. While this appears to make development easy, it creates other issues:
Developers working on the same application get compartmentalized into different project contexts, with little collaboration and synergy. There are instances when developers being unaware of similar requirements in other projects fail to exploit or optimize each other's code.
With projects of different duration and start/end dates, merging code branches, conducting regression tests, and resolving code-merge issues often becomes a major exercise and challenge that is not well planned and budgeted.
There doesn't seem to be a defined way to help different project/sprint teams who are working on the same application to collaborate and manage. Development teams evolve their own methods with ad hoc meetings between teams to sync up on different project backlogs, leaving code-merge decisions for the end of the release.
Figure 3: Application Scrum
An Application Scrum would help teams that are working on the same application to collaborate with each other. This not a true Scrum; instead it is a sequence of sprints where the sprint backlog is derived from the sprint backlog of projects impacting this application. This model requires the following practices:
Figure 4: Align sprints across projects
Align the sprints of all projects to the same start and end dates. While different projects in an organization do have different start and end dates, the proposal here is to align the sprints across projects. For example, an organization that adopts three-week sprints can mandate that a sprint can start only on week 1, 4, 7, etc., of the year. There are multiple benefits at the application level in aligning all project sprints:
Application sprint backlog
Code branch: All code branches reach a logical end at the same time. This may help you consider a single code branch for all projects, or at least minimize the number of code branches. If you do have multiple code branches, aligning gives you a chance to merge branches and maintain overall integrity more frequently -- perhaps at the end of each sprint.
Cross-project backlog review: You can aggregate and review backlogs items of other projects pertaining to a specific application and find ways to optimize and prioritize better.
An application sprint backlog is the union of all sprint backlog entries that pertain to this application. With sprints of all projects being aligned, at the start of a sprint compile all sprint backlogs, segregate the backlogs by the application each impacts, and derive each application's sprint backlog. So while the project backlog is a list of changes to be made to different applications, the application backlog is a list of changes to the application required by different projects. This gives engineers assigned to an application a consolidated view of all changes to be made. Rather than looking at it as several unrelated projects, it is now viewed as one consolidated backlog, giving the team a chance to discuss and distribute tasks more effectively. This consolidated view tremendously facilitates collaboration among the developers and subject matter experts of an application.
A daily stand-up meeting at the application level and another at the project level is overkill. But both are required, and perhaps the application-level meetings can be less frequent. Depending on the number of different projects impacting the application, a stand-up meeting perhaps once or twice a week to review the overall application backlog status is sufficient. It keeps all team members informed and creates synergy and co-ownership of all the changes being made to the application.
Scrum practices increase collaboration during project execution. Scrum works best when the items in a product backlog relate to a single product and the team working on this does not have to compete with other teams making changes to the same product. In large organizations, extra care is required to ensure that collaboration at the lower application or service level is also addressed. Aligning sprints and defining a second layer of Scrum at the application level will help.