I recently coached on an R&D project that involved a lot of research work. The team reviewed the state of the art on new problems; read and tested papers; found and tested completely new ideas; prototyped a lot, sometimes digging deep into a possible solution only to realize eventually that the whole approach was broken and starting over; and so on.
The project consisted of delivering a big image processing engine, with state-of-the-art algorithms. The engine would be delivered "as is" to customers or integrated into desktop applications. Individuals on the team had been introduced to Scrum in the past, but Scrum did not look like a good fit for everyone. Googling and reading books for help did not really give us much advice about Scrum for research — it felt as if we were among the first ones to do it. That’s why I hope some may find it helpful to read about the adjustments that we made.
1. Embracing uncertainty
When a Scrum team must determine whether they can implement a user story right away or they need a spike first, they evaluate the level of uncertainty. If that level is above a certain threshold — let’s call it the spike threshold — they’ve subconsciously set as being an unacceptable risk to take into a sprint, then they go for a spike first. While I admit it is hard to evaluate, let alone compare between teams/products, my feeling is that this spike threshold was quite stable across the teams I had worked with. It was such that teams had to start with a spike on approximately 10% of their user stories. And this worked fine for them.
On this particular project, using the same threshold would have led to more than 50% of our sprint backlogs as spikes, which we didn’t want because that would delay our ability to deliver value to our customers. Therefore, we agreed to accept more uncertainty in our sprints by increasing our spike threshold so as to keep the amount of spikes in our sprints fairly low and get a chance to deliver value faster.
Note that even so, we had more spikes in our sprints than most Scrum teams do.
As a consequence, we had to accept uncertainty in our estimates and forecasts. We had to accept that we would have to put an estimate on stories even though a fair share of the work to be done was yet to be discovered. We mitigated this by having a good baseline for story points estimation
. We reviewed and improved it over and over again.
By embracing uncertainty, what we are also embracing — other than a high standard deviation on our velocity metrics — was plain failure! We accepted that sometimes a story would be trashed after a few days of work. And we’d have to figure out, mid-sprint, what to do instead.
Embracing failure is hard for everyone, but it was even harder for the people on that team. PhDs and researchers don’t like failure! They like to think that they have "thought about everything" before actually implementing anything, and when they start, they like the process to be flawless.
2. Interruption buffer
Another adjustment that we made was a documented Scrum Pattern on ScrumPlop and one of Jeff Sutherland’s favorite ones — the Interruption Buffer
Following are differences from the way I had been using it in the past:
- It was bigger (about 40% of the team’s velocity). Yet I know that some teams have an Interruption Buffer that’s even bigger than that, so it was not too exotic, I suppose.
- We didn't use the buffer to handle incoming bugs and change requests but rather to add user stories to a sprint, following important discoveries made by the team during the sprint.
For example, if our velocity was 30, our sprint backlog would look something like this:
When a spike would be done and we would have acquired that new knowledge, we would discuss and decide what to do next, knowing that it could not be bigger than 12 points. While I would not recommend that in general, this was a better option for us, in this context, than having to wait for the next sprint planning before we could continue on that topic.
- An 8-point user story
- A 5-point spike
- A 2-point user story
- A 3-point spike
- A 12-point buffer
I must say that this tweak really was a major step forward in making everyone on the project realize that Scrum was actually fit for research. Until then, the team misunderstood Scrum as "Defining 100% of the work for the next iteration every three weeks; and making it a crime to make a mistake, to change your mind, or to discover new things in between sprint plannings."
3. Becoming pros at splitting user stories
While it is never easy for new Scrum teams to split their user stories, a little bit of Googling for tips, plus some coaching from their ScrumMasters, usually does the trick. And once they have split their epic in to two or three parts, they are already small enough that you can safely put them into a sprint.
But what this team worked on was harder to split. We sometimes had to go through four cycles of splitting an epic before user stories were considered small (more on "small" later). So I took advice from the best by reading Craig Larman and Bas Vodde’s books Scaling Lean and Agile Development
and Practices for Scaling Lean and Agile Development
Larman and Vodde present a series of techniques that we had to get familiar with. Moreover, they helped us realize that it didn't need to be perfect, but that whenever perfection was not possible (spoiler: often), you had to choose carefully what to prioritize. Would you rather have something big or something dependent? Would you rather have something that’s not testable or something that brings no value?
For the team to become familiar with these concepts, I organized a workshop in which, after presenting the various splitting techniques from Larman’s book, I gave small groups a couple of big user stories and asked to split them. Each group had the opportunity to experiment with the various splitting techniques and to discuss the trade-offs of each one in concrete terms. Splitting the work was also something we improved on over time, as it was a recurring topic at our sprint retrospectives.
4. Redefining "small"
Up to this point, I defined a "small" user story as taking up to 50 hours. I would tell the teams I coached to aim for that. Here, it simply did not work. The uncertainty was too high, the domain (and the code) too complex. In fact, as an experiment, we once started a new feature of our engine by implementing its skeleton — a picture-perfect implementation of nothing, with the full test coverage, as we wanted it according to our Definition of Done. That skeleton took us exactly 30 hours to deliver. So, if 50 hours is your maximum, that leaves you with 20 hours to actually think about and find a solution for whatever problem you have, and code it. Instead, we agreed on another definition of small: "a pair of individuals for half a sprint," which totals 120 hours. That’s what we aimed for, and that still was a challenge.
As you can see, this is nothing too fancy. And it’s interesting to point out that this was absolutely 100% Scrum. So when I said that we made some "adjustments," let’s be clear that these weren’t adjustments to Scrum
. These were adjustments to how I usually help teams implement Scrum. It’s different.
I am still very interested in hearing more about Scrum for research. So should you have some advice, please share below!