Get certified - Transform your world of work today

Agile Project Reporting and Metrics

07/29/2013 by Saji Pillai

Agile processes are generative, not prescriptive. Processes need to evolve as needed, not be prescribed up front. A prescriptive approach generates complex and complicated processes, whereas a generative approach begins with a set of simple processes and adds others as they are needed.
-- Jim Highsmith, author of Agile Project Management
I have written this article to address an issue I've recently run across in my work: The PMO is finding it difficult to adjust to the Agile process, and this is affecting the reporting of the R&D team. Management is in the process of moving the ALM (application lifecycle management) process into more of traditional process to align with the PMO. I would like to suggest an alternative approach.

To give some background: Agile recognizes that most effective software processes cannot be defined up front; it is a continuous process. Traditional ALM (application lifecycle management) processes focus on defining and enforcing processes and spend very little time identifying and delivering what customers need. Although their aim is to improve consistency and quality, this often comes at the expense of productivity and delivery. Traditional ALM has also uses monolithic, heavyweight tools, causing fragile builds and ineffective hand-offs between development, testing, deployment, and operations.

Agile ALM frees companies from heavyweight processes and tools. Using continuous builds, the customer regularly receives working software that allows the customer to gauge the product and provide corrective measures before it is finalized. While traditional ALM applies tools to enforce predefined, standardized process, Agile ALM is the application of tools to support people and the processes that best suit the way they actually work when creating great software. 

A typical product development company has an issue management process (ticket handling for bugs/defects), change/feature management processes, and new customer implementation processes. The PMO is a centralized resource for all the IT project teams, monitoring project delivery, with a wide range of projects with defined goals and time frames. The PMO is responsible for resource planning, including staff competencies, assignment, and capacity planning; metrics and reporting to management; and definition and standardization project implementation and delivery processes, guidelines, training, and other project support services. Project managers need to consistently communicate about and report on their projects across business units and development methodologies to the PMO, so the PMO can present a single dashboard across a breadth of projects within a COO's control. It is not surprising that the PMO is often heavy-handed, imposing a strict approach to metrics and guidance across all projects.

Among the above responsibilities, the greatest impacts on the Agile project team are:

  1. Providing the dashboard to all stakeholders
  2. Providing metrics for processes, project deliveries, etc.
  3. Defining standard development practices that will support and can be easily mapped to traditional project reporting and practice
Agile processes must exist within the PMO infrastructure. This requires that the Agile team be willing to conform (at least externally) to traditional project management practices. Not only must the Agile project adapt but the PMO must as well. Here there's a benefit from the ability of Agile teams to accurately measure the status of a project and adapt to changing requirements.

Legacy PMO tracks projects on the basis of project schedule, budget, issues, and risk. This data is largely fed through project management tools (MSProject, etc). The PMO tracks time and cost estimates from project task dependencies across the phase of the project life cycle. Agile teams must communicate effectively, by using language and metrics that the PMO can appreciate. Agile teams report on velocity (the rate at which the team delivers a story), burn-down charts (the trend of remaining effort), test coverage, and running test features. These do not effectively help reporting on time, budget, and risk analysis. Hence the Agile team needs to identify and bridge this reporting gap.

Sample Agile reporting

Release in Agile ALM mapped in the project plan:

Release backlog:

Traditional project plan and mapping user stories to the project plan::

Metrics such as earned value analysis (EVA) and cost to complete are certainly valuable to all stakeholders.

Key earned value management terms defined

(Note: The term value in EVM does not refer to business value. The term refers to value as expressed by the project or program budget.)

Planned value (PV): The value of the work planned to be accomplished based on the budget (in dollars or hours)

Earned Value (EV): The integrated value of work actually accomplished based on the budget (in dollars or hours).

Actual Cost (AC): The cost incurred for that increment of work.

Budget at Complete (BAC): The budget assigned to complete the work.

Estimate to Complete (ETC): The forecast amount to complete the remaining work (in dollars or hours), based on past performance.

Estimate at Complete (EAC): The forecast total amount for all work in the project plan, based on past performance.

Formula Metric Analysis
Planned Value BAC * Planned
Percent Complete
The planned value indicates how much value was planned to have been generated by a particular milestone or point in time.
Earned Value BAC * Actual
Percent Complete
The earned value indicates how much value has actually been generated at a particular milestone or point in time.
Cost Performance
Index (CPI)
EV/AC This metric indicates how many cents have been "earned" out of every dollar spent. It measures cost efficiencies.
Performance Index
PV/AC This metric measures schedule efficiency. It indicates how fast you are
progressing against the rate of progress planned.
This metric is the forecast amount to complete the remaining work.
Forecast cost for the total planned work.
Agile story points can be used as the measure of work planned and work performed, providing the basis for all of the traditional earned value calculations. Percent complete can be derived by dividing the number of complete sprints by the total number of planned sprints. Actual percent complete can be derived by dividing the number of story points completed by the total number of story points planned for a release. To calculate the initial baseline in Agile, five data points are required:  
  1. The number of planned sprints in the release backlog
  2. The length of each sprint in calendar days
  3. The number of story points planned for the release backlog
  4. The budget planned for the release backlog
  5. The start date of the project
To calculate EVM in Agile:
  1. Number of sprints completed: This measures the expected percent complete when divided by the total number of sprints planned.
  2. Net story points added (this can be a negative number if story points are removed from the release.) This reflects the net change in work planned for the release, in effect re-baselining the release. Actual cost to date at each boundary: We are measuring results to date, and therefore we use the cumulative actual cost as of that sprint boundary.
  3. Cumulative story points completed, which measures the total amount of work completed for the release as of that sprint boundary.
Agile dashboard for traditional PMO
Budget at Complete (BAC)   The planned amount you expect to spend
Actual Cost (AC)    
The actual cost to complete the work
  Planned release story points for the release; story points are defined at the product backlog level
Expected Percent Complete (EPC)   Current sprint(n) / Total planned sprints
Actual Percent Complete (APC)    
Story points completed / Total planned story points
Planned Value (PV)   PV = BAC * EPC
Earned Value (EV)   EV = BAC * APC
Cost Performance Index (CPI)   CPI = EV / AC
Schedule Performance Index   CSPI = EV / PV


Team dashboard using a task board
In Agile, it is a common practice to maintain the task board on a wall to help everyone visualize the project status. It's a visual process management system that tells what to produce, when to produce it, and how to produce it.

The project status as a dashboard

Defect metrics as per release backlog