Why We Dropped Scrum for Kanban in Our Lead-up to Launch

29 May 2013

Brendan Marsh
Spotify

Let me set the scene: It's around three weeks before the launch of our product (an expensive, big-bang PR launch) and the team is discussing the plan of attack for launch. We're coming up to the end of our sprint and grooming the backlog so that it's ready for the next sprint, and we're also releasing the product early to a closed group (media/bloggers) so that we can get some early feedback. Apart from a couple of features, our last sprint is mostly related to compliance and legal requirements for launch, with the rest being bug fixes and polish.

At face value, the plan would be to complete one last sprint and then enter maintenance mode. We have also imposed our own change-approval process, to ensure that we don't break the platform or any legal requirements.

But there are a few difficulties we face in the lead-up to launch:

  • Our developer responsible for deployment, systems admin, and dev ops will be away on holiday, and therefore the team will collectively need to pick up his responsibilities. (Our initial solution was to factor this learning process into the backlog as a sized story to be included in the next sprint.)
  • Media influencers using our platform early will be providing feedback, some of which may teach us something we don't know that's critical to a successful launch.
  • There are some external "parties" we're still waiting on for various reasons, which may mean that we need to pull tasks into the sprint — which, in Scrum, is a no-no!
  • There is a lot of work to do before launch, some of which is absolutely critical and some of which is very nice to have. But stakeholders will be pushing to have all of this ready for launch. Now is certainly not the time to be overcommitting during sprint planning and increasing the risk of bugs, particularly due to the high degree of unknowns that can happen so late in the game.
  • The team is already working long hours to meet the high demand of features and stability for launch. If they keep this up, quality is going to drop at a critical time. We need a healthy and motivated team to make sure this all goes according to plan.
  • The next sprint falls on our Easter long weekend. I suggested that we avoid the assumption that the team will work over this holiday. If people want to that's fine, but I don't want to base the length of the sprint on this assumption; it's not fair and could demoralize the team.

So, at our retrospective, the team raises all of these very legitimate concerns. We could continue with the next sprint, but we'd inevitably be breaking the foundations of Scrum as soon as anything went wrong, or as soon as critical requirements came in.

So how do we:

  • Reduce the risk imposed by losing our developer who's in charge of deployments.
  • Skill up all our developers in this process.
  • Be able to respond to constant change on a daily basis (i.e., breaking the usual timebox constraint) that will come from third parties, media influencers, testers, etc.
  • Still do the necessary work prior to launch that is vital for compliance, legal, and stability reasons.
  • Restrict work in progress (WIP), keep working hours at a sustainable pace, and keep morale high.
  • Keep quality at a high level.
  • Last but not least, remain Agile, pursue continuous improvement, and stick to the inspect-adapt feedback loop.

Our answer: to (temporarily) ditch Scrum and employ the principles of Kanban:

  • Visualize.
  • Limit WIP.
  • Manage flow.
  • Make policies explicit.
  • Implement feedback loops.
  • Improve collaboratively, evolve experimentally.

To follow these principles, we created a Kanban wall and mapped out the ideal process for iterating toward launch. It involved testing each feature in multiple environments and deploying daily. Deploying daily is time-consuming, but it allowed all of our developers to become involved in the deployment process, which decreased our risk of deployment issues (better we find out now than later) and increased the confidence of the team that it could achieve a smooth launch.

To ensure visibility of bottlenecks early, we imposed limits on most steps of this process. Almost immediately, we were able to identify QA as a bottleneck and bring on a new resource to cover the increase in testing needed to follow this process of testing multiple times in different environments.

In lieu of a daily stand-up as mandated by Scrum, we held a daily catch-up with the product owner in which we scanned (and groomed if necessary) the backlog and discussed our flow. Specifically, we asked:

  • How is progress coming along generally?
  • Is this process (Kanban) still working? What's working and what needs to be improved?
  • Are there any bottlenecks? Do our WIP limits needs to change?
  • Are there any impediments? (We even created a column behind the backlog that had a WIP of 0 for "blocked" tasks, which created visibility and alarm as soon as there was an impediment.)
  • Are there any new updates from the product owner?

At the end of each day, the updates to the product were deployed to the production-ready environment and tested by a tester in another time zone (we have staff here in Sydney and the U.K.). This meant that the product owner had a list of tested features/tasks sitting in his email inbox each morning. To save time, only features that were demonstrable and new were physically demonstrated to the product owner when they were fully completed, saving us time in lengthy Scrum review sessions.

Finally, I managed to borrow a projector and display the Kanban wall (we're using Jira/Greenhopper) onto the wall opposite our developers. As soon as a WIP limit was breached, the column became bright red.

It's been about a week since we implemented this process, and it's been really effective. My greatest concern was fear of rejection from the product owner, because he effectively loses his typical "sprint commitment." But he gains the flexibility to adapt to changing priorities on a daily basis leading up to launch.

Honestly, I thought that this was going to be an issue, but to my surprise he has (voluntarily) expressed satisfaction with this process on multiple occasions and loves the fact that he is seeing progress from the team on a daily basis. For him, it's a much less nerve-racking experience each day we get closer to launch. We've been tweaking this process since we started and we're getting better every day.

This article is by no means suggesting that Kanban is better than Scrum. Rather, in these circumstances, within this period of time (about 3 weeks before launch), it was better suited to handling the chaotic nature of iterating toward launch. I hope that we can switch back to Scrum once our product has launched and we're considering new stories.

Post-launch update

It's been more than a month since I wrote the first part of this article, so I thought I should provide an update, now that we have switched back to Scrum again, completed our first sprint, and are halfway through the next one.

We launched a product called Jaro, a social-good project with a rather ambitious goal of raising $1 billion for charity and naming one lucky winner of a tournament-style game. The launch was successful and the software we built worked and scaled as expected. Shortly after the first launch, we were featured on BBC Click on the BBC World Service, being transmitted all over the globe (and smashing our servers with traffic).

We had focused on only the most important tasks each day in the lead-up to launch. This acute, daily focus on what was most important helped us inspect and adapt daily, to ensure that the team was only concerned with what was important now. For the most part it worked, but it wasn't sustainable. Shortly after we launched, we conducted a "launch retro," in which we discussed what was working and what wasn't.

While the team liked the Kanban process, we reached the conclusion that it wasn't going to be sustainable, as it was harder to measure and understand our capacity, and it was difficult for the product owner to give informed estimates on when features would be delivered. I have no doubt that we could have improved it, but going back to Scrum seemed like the easier, safer option.

So, in true Agile fashion, I'll reflect on what's working for us now that we're back to Scrum and what we're working to improve:

What's working: Fewer hours, less stress, better quality. I previously mentioned that the team was working long hours (and while we had some mild improvements through the Kanban process, we ultimately didn't improve this as much as we'd liked). Well, now the team is generally coming in on time and leaving on time, which has also had the knock-on effect of improving quality as well. Our Definition of Done is being followed, and we're also working toward building some more front-end automated tests.

I attribute all of this to sprint planning and the sprint commitment. The team really understands the importance of sprint planning in terms of being in control of how much they commit to and the effects that will have on work over the space of two weeks.

What we're working to improve: Release schedule. When we were using Kanban, we were continually releasing to our dev, then staging, then production environments on a daily basis, which meant that our early adopters, charities, and partners could see their requirements being fulfilled almost immediately. This was vital in the lead-up to launch (particularly when getting sign-off/approvals). Now that we're live, the team is getting asked to release patches/small changes mid-sprint, which isn't exactly "by the book" when it comes to Scrum.

To give us visibility around the extra time it takes to do this, we've factored in a story-point buffer whenever it’s likely that we'll need this done. We're also factoring in our deployment testing process when it comes to considering a story "done," as this was favorably received during the Kanban process.

So why should you consider Kanban over Scrum?

From my experience here (and yours will no doubt be different), Kanban is useful when requirements and priorities change quickly and often. If we truly value the principles of the Agile manifesto, then we should welcome changing requirements, even late in development; but in my opinion, Scrum is only good at this when we can handle changing requirements every one to two weeks (I avoid saying four weeks!).

This is why Kanban is the unofficial preferred framework for maintenance staff, as it is more concerned with limiting work in progress than it is fulfilling a release plan/backlog.

I hope this article helps you to make a decision on whether this approach is right for you. I'd love to hear your comments, experiences, and feedback on this approach. I’ll do my best to answer any questions you have as well.


Opinions represent those of the author and not of Scrum Alliance. The sharing of member-contributed content on this site does not imply endorsement of specific Scrum methods or practices beyond those taught by Scrum Alliance Certified Trainers and Coaches.



Article Rating

Current rating: 4.8 (5 ratings)

Comments

Bronwyn Calleja, CSM, 5/29/2013 1:15:58 AM
Thanks for the article, it was really interesting as a newbie to Scrum leading up to our own pre-launch phase. We were intending to stick with Scrum, perhaps change from a 2 week sprint to 1 week. My concern with Kanban are:
1. that our product owner does not have the knowledge or time to be able to prioritise bugs on a daily basis
2. our team is also geographically distributed over 3 main countries (with a few others thrown in as product team SMEs)
3. our product team is not experienced with Kanban.
So, I'm thinking Kanban may not work, as even though our dev team is experienced in Kanban, it relies very heavily on the Product Owner input.

I guess my questions are:
1. did your product owner know much about Kanban before going into it?
2. I'm assuming it required more of their time than the Scrum process?
3. Do you think having a Product Owner offset 12 hours to the Dev Team more, less or equal difficulty level in Kanban than if using Scrum?

Thanks,
Bronwyn
Brendan Marsh, CSM,CSPO, 5/29/2013 10:44:06 PM
Hey Bronwyn, thanks for your feedback. I'm not sure that your PO will need to be any more knowledgeable to work on a project in Kanban, ours didn't really need to learn anything other than to know that we'll be taking whatever is on the top of the backlog each day (Pull mechanism). Having a team geographically distributed will make things hard, as the ability to communicate is diminished, it must be difficult for you! None of us were particularly experienced with Kanban, but we started simple, analysed our ideal 'flow' and set our own WIP limits, then gradually improved the process each day. Doing this over telepresence is going to be hard I'll admit. But you should still strive for anything that will give you feedback quicker in a stressful, ever-changing phase like pre-launch.

Answers to your Questions:
1. Our Product Owner is a trained CSPO, but as far as Kanban is concerned, no not particularly. He was still using the same tools to manage the backlog. He had to spend more time with the team, but everyone agreed that this was beneficial at such a crucial stage.
2. Yep, but it was a worthwhile tradeoff
3. Not sure I understand what you mean, but I assume you're saying "is Kanban easier or harder if your PO is in a 12 offset time zone?" If so, I'd say its probably harder, but it depends on how much faith the PO has in the team to help manage priorities & the complexity of work being done.

Hope this helps! Good luck Bronwyn :)
Bernard Boodeea, CSM, 5/30/2013 8:00:35 PM
Great article! This is a great example of an agile leader.

It looks like you had to change from Scrum to Kanban because of the constant change of requirements before a launch which had a fixed date.

In Scrum, a lot of projects suffer in the last sprint because new things come up from different source and we cannot add a new sprint due to the deadline.

Thank you for sharing this with us and this will definitely help to consider using Kanban if there is no choice but to have lots of changes prior to a launch. Even though it is against the principle of Scrum to make changes during a sprint sometimes it is hard when you have a PO who does not understand or does not want to understand the destructive impact.
Jurriaan Kamer, CSM, 11/19/2013 7:36:02 AM
Great insights Brendan, thanks for sharing.
Shawn Tretter, 5/26/2014 2:31:05 PM
This is a great article Brendan! Thanks for sharing with the community.

I am new to scrum, but was curious if you believe there could be a hybrid scrum methodology in the sense of continuing with scrum best practices/requirements while adapting a Kanban approach?

Do you think the ideology of scrum can be modified to be able to be used during the rapid closing times with maintenance and ensuring no burnouts from the team?

Thanks again for sharing and I look forward to your feedback.

You must Login or Signup to comment.