Can Hours Spent Help You Understand the Efficiency or Productivity of a Team?

14 April 2014


Traditional project management endorses hourly estimation of every planned task. Managers typically track planned versus actual hours as a benchmark of project progress. When the actuals don't conform to the planned hours, it reflects on the efficiency and performance of the team and the individual.

In Scrum, we accept that accurate estimation is not possible on high-level stories, and so we use story points. Yet during sprint planning, most teams break down the story into fine-grained tasks, and then they go right ahead and put hours against these tasks. Several product-based companies (at least in India) hire project managers (QA/engineering), predominantly from service-based backgrounds, who use these hourly estimates to determine the efficiency or the productivity of the team. When a "resource" (their term) spends more hours than planned, they are deemed low-performing resources who can be replaced. If some resource is taking much less time than the planned hours, then he or she is deemed a high performer who deserves credit.

Due to these practices, developers live in constant fear of slippage of hours. In order to avoid slippages, most developers add a cushion called a buffer into every task. If the industry standard is a 15 percent buffer (on the epic), developers tend to make it 30 percent. This gives them enough time to complete their task within the planned hours.

The following shows an actual plan that was proposed to a PO after adding all these buffers:
Buffered Release planning
Why development effort worth 5 sprints was proposed as 8.5 sprints by the team

As we all know, accurate estimates are rare because domain, technology, complexity, and depth of projects cannot be fathomed beforehand. However, because hourly estimates are taken as performance metrics, annual increments and promotions are directly linked with these metrics. When people do not perform (meaning they take more time to complete a task than planned), their annual compensation stalls, and so does career growth. The fear of not performing drives teams not to take any risks while developing and to always keep good amount of buffers in the story.

During sprint planning, when we as team members ask our colleagues about how we can reduce the time taken to develop a particular story or why we need the amount of time that we are providing, there are always vague answers such as, "Maybe you can do it in less time, but I need this much."

When I was in engineering, we heard or spoke those words with our peers many times. All these excuses are because of the fear of losing, which makes us take a backward step. When we continuously miss our planned hours, our managers tend to believe we are not performing.

Another trick that developers use is to not log the actual hours that they spent on a task or a story. Let us consider a task that is demanding and takes more time. During planning, the team felt that this task might take only 20 hours. While executing the task, the developer is spending more time by staying late for the day. However, while logging the time sheet, the developer just enters the minimum productive time (say 5 hours) for the day. This gives them 15 more hours for the task. Upon completion (let us say 40 hours), the time sheet actually has 20 hours accounted for and the remaining is unaccounted.

This way, developer performance is constant but the actual hours spent will look like this:

Sprint progression in terms of actual hours spent

You can find this kind of graph in many service companies that bill every hour to the client.

What are we proving with this?

With these methods, are we doing justice to the Agile way of development? That is in serious doubt, because Agile is all about transparency and openness. It gives us the maturity to accept mistakes and improve upon them. When we present graphs like this, we are not doing justice to our work or to our organization or to our clients.

Agile speaks about doing the right thing in the right way to accomplish the goal. Masters of Agile principles always advocate to not micromanage everything and accept the fact that "It will take as much time as it needs to be done." This is the main reason why we do story point estimation, which is completely relative.

Is it necessary to have hourly estimates for tasks?

Do we have any artifact that uses the tasks of a story? The only chart that defines the productivity of the team is a velocity chart, which does not consider tasks. All it considers is how many stories we complete in a given time. When there is no artifact that requires the number of hours, do we actually need to use hours in our understanding of efficiency?

Knowing the number of hours worked by a developer does not help us understand how productive and valuable the developer is for the team. It just gives us the information that the developer worked 40 hours a week on the project. This may be useful if you are billing the client by the hour.

Does this give any information about what the value is that the developer produced within that time for the project?

Not doing the right thing does not add any value to the story, which the developer might not know. Penalizing them for this is not a good way to motivate the team. When managers do this, it affects the developers in negative ways, such as:
  • Not being transparent in accepting mistakes
  • Trying to defend themselves
  • Finger-pointing at others
  • Never trying getting help from others
  • Never taking ownership
These negative traits may not impact the developer immediately, but they adversely impact the team in areas including:
  • Low productivity
  • Team morale
  • No ownership of what they do
  • Hiding the ant until it becomes an elephant
  • No communication on the issues they face
The mother of all these evils is due to the wrong metrics based on hours worked.

How do we improve the team's efficiency without hours?

In the organizations where I have worked, we did use hours for the tasks, but we never worried too much about them per se. We mostly relied on just two status markers for any task: planned, done. However, we mandated to ourselves that during sprint planning, we would commit only what we could deliver in the sprint. Nothing more, nothing less.

This change helped the team realize that the actual hours are not relevant, and that what is relevant is that is the standing up to the delivery. Teams started delivering on all the commitments and, in many cases, they started taking more once they completed the initial commitment. This actually increased velocity. Many times during stand-ups, developers openly acknowledged that they worked X hours more than they thought they would beforehand and that they needed some help to finish the work.

Though developers stayed back to finish the work, there was no single instance where they entered minimum hours of work. They entered how many hours they worked that day on a particular story without worrying the total time pending. This helped us to get the reality on the story, with accurate updates for all stakeholders and no last-minute surprises.

This approach helps teams remain collaborative and solve their problems themselves. During our retrospectives, developers discuss on what went wrong with a particular story and how we can correct it. There is no finger-pointing and everyone is able to accept the hard realities.

This helped us to nail the problem in its initial stage.

If hours are not used to measure efficiency or productivity, what else can be used?

A lot of negative traits of software development can help improve the productivity of a team if used positively. Following are the traits we used:
  • Number of bugs in the system
    • QA specialist: How many bugs did you find early, even before the build, that were fixed by developers? How many very critical bugs were found after the build?
    • Dev. specialist: How many were avoided in the second run and fixed immediately? How many were critical? (Use this positively, not to penalize.)
  • Test scenarios and use cases covered
    • QA specialist:
      • How many valuable scenarios/use cases were written to cover the story?
      • How many critical scenarios were identified and shared with others early during the sprint?
    • Dev. specialist: What was your contribution in developing those scenarios/use cases?
  • Quality of code
    • How good was the code? What are the improvements seen over a period?
    • How good was the code coverage using the unit testing?
  • Refining requirements
    • Dev. and QA specialists: What is your contribution in refining the backlog by adding details to the requirements?
  • Number of tasks required for a story
    • Dev. and QA specialists: How many tasks are actually required?
  • What was the difference between planned stories and completed stories by the end of the sprint?
  • How did the team collaborate to solve the issues?
  • How vocal are team members regarding a problem?
  • How approachable are the developers?
  • How committed is the team in achieving the goal?

How do we do an appraisal at year-end?

Appraisals in Agile-based companies are still a challenge, and most Agile ScrumMasters prefer to do it equally as a team. As a ScrumMaster, based on your analysis, you can always identify people who contributed more than they needed to. Recognize them for their efforts. This will motivate others. The following are some ways I can recommend to identify an individual who outperformed consistently.

Option 1: Chocolate game to identify an individual star
This was a game taught by our grandmother during our childhood days.

Requirement: 70 Chocolates (for a 7-member team), team members, ScrumMaster
Rules:
  1. Everyone should distribute all 10 chocolates to others.
  2. A member can distribute the chocolates to one or more than one person.
  3. All chocolates can be given to one individual.
The game
  1. Distribute 10 chocolates to each member of the team.
  2. Announce the rules.
  3. Ask team members to consider who helped them a lot, and distribute the chocolates.
  4. Once all members have handed out the 10 chocolates, stop the game.
  5. Ask everyone to count the number of chocolates they received.
  6. Announce the winner.

How does this translate to performance appraisal? This game actually helps the ScrumMaster identify the clear favorite of the team. This game can be considered as another model of 360-degree feedback that is used to appraise an individual. When this individual gets a slightly better hike than all others, no one will feel bad because this individual was selected by the team, with no external pressure involved.

Option 2: Sprint sheet as benchmark
In Agile, everything is visible through the sprint backlog. As ScrumMaster, you can review the sprint backlog to identify the individual who contributed more to the team. Also, having an eye on the measurements that we used to calculate productivity and efficiency can bring out a clear favorite during a particular duration.

Conclusion

Using these options diligently will help motivate the team and encourage them to roar ahead with positive energy.

A happy team means a happy customer, so make your teams happy.


Opinions represent those of the author and not of Scrum Alliance. The sharing of member-contributed content on this site does not imply endorsement of specific Scrum methods or practices beyond those taught by Scrum Alliance Certified Trainers and Coaches.



Article Rating

Current rating: 3.7 (3 ratings)

Comments

Stefan Jossen, CSM,CSPO, 4/14/2014 2:01:58 AM
Thank you for your story.
Estimating user stories with story points and task in hours breaks the agility of scrum.
Tim Baffa, CSM, 4/14/2014 9:41:42 AM
I agree that a relative, totally subjective measurement like story points directly relates to the anticipated productivity of a team. I am a proponent however of the team decomposing a story into potential tasks during refinement, and especially as part of Sprint Planning Part 2 (with rough time estimates) when the team determines their capacity verus the Product Owner offer made during Sprint Planning Part 1.

It simply doesn't make sense to base a current sprint on historical story point velocity if team capacity is reduced due to vacations, holidays, or organizational demands outside of the sprint. The capacity planning must take place, and in all likelihood, the team will understand all of the tasks needed for a story by the time they reach Sprint Planning Part 2.

The key here is to relate to the scrum team as a "team", and to move away from the traditional individual-based performance measurements. The team is responsible for sprint delivery. The team is responsible for all estimates made. Estimates should never be made based on one individual's possible productivity, because there is no guarantee that a specific individual will perform specific work within a sprint. The team is responsible, and having work completed by someone not as adept as another team member only serves to strengthen the team as a whole for future work.

I disagree with singling out any team member as more high-performing than other team members, since this simply feeds into prior workplace dysfunction and does not promote the team growth needed to function as a single unit in Scrum.

Scrum is all about teamwork and how the team can better serve the organization. Being a Scrum Master is all about serving the team and helping them grow into a highly-performing unit of the business. Measuring individual performance is a very slippery slope, as it always has been.
Albert Arul Prakash Rajendran, CSM,CSPO, 4/14/2014 12:08:14 PM
Completely agree with you Tim, Agile is all about team work not individual. however, In many Indian development scenarios, it is still in a Individual mind set where managers appraise individuals (hard reality) than the team.

When we ask the organizations to move away from traditional individual based performance measurements, their question back is, then how do we need to do performance and also identify those who can be moved up the ladder (Unlike US, In India, if you are not moving up the ladder for every two years you are considered as a under performer).

Due to this performance aspect, though team is responsible for the delivery, estimate, planning etc, they tend to add more buffers to a bigger extent.
Zach Bonaker, CSP,CSM,CSPO, 4/14/2014 12:10:13 PM
@Stefan: How does the use of points for stories & hours for sprint tasks "break the agility of scrum"?
Charles Cain, CSP,CSM,CSPO, 4/14/2014 1:36:53 PM
Good article Albert.

In your reply to Tim, you say that the team tends to add more buffers due to the performance aspect. While that is the case usually, one way that you can minimize that is by the use of the retrospectives. When the team begins adding buffers, you can query them and say "isn't this something like task A that we did back in Sprint 5, or am I misunderstanding the story here?". That way, the team knows that you are watching their estimates. By challenging this you can minimize the amount of buffering that happens.
Jem Djelal, CSM,CSPO, 4/15/2014 2:23:02 PM
@Zach,interested to know based on your question...

do you see a benefit for breaking down stories into hours of work?
Deepak Srinivasan, CSP,CSM, 4/16/2014 1:01:09 PM
@Jem - I believe Mike Cohn advocates such an approach in transitioning to agile. I have noticed that some people advocate completely eliminating estimation by using roughly similar story sizes. I had never thought about it, but it certainly seems like a very effective idea. I find that the estimation process is one of the biggest areas of contention between the team and the stakeholders. It creates large areas of mistrust. However...back to the point, if teams have already been estimating in hours/days and have been successful at it, there is no reason to change that. Why not pick other areas to improve in the transition backlog.

You must Login or Signup to comment.