What is the most effective way to measure a software development team’s productivity? On an Agile software team, where we are ultimately delivering value in the form of functional software, how can this be accurately measured?
This was the area of discussion the other night at the MN Scrum Experiences user group. It was quickly noted how any metrics can be gamed to skew the measurement results. While this remains true, many folks implementing Agile still get asked the question of how to measure whether an Agile or Scrum implementation is effective.
Some of the common measurement metrics for a Scrum team include:
- The number of story points completed versus those started
- Team velocity (the number of story points completed plotted over time)
- The number of defects (e.g., opened versus closed in the Sprint)
Other team metrics that were discussed that teams use included the amount of unit test coverage and the percentage of test cases run that pass.
All of these metrics can provide just as much value as they can do harm. A team metric should be used as an evaluation of the team, with the understanding that disruptions in the team will affect the metrics. And what do these measurements do about determining if a customer is satisfied with the process of software development?
One takeaway that I had is that it is important to develop a good measurement for customer satisfaction. Fred Reichheld describes this concept (customer delight) in his book The Ultimate Question (I’m adding that one to the reading list).
Ultimately, if a Product Backlog is prioritized by business value, and if the Agile team continues to deliver successfully from the backlog as it is prioritized, the velocity and number of defects resolved provide a pretty good measurement. So long as in the end the customer is also delighted, and that the metrics are not used out of the context of measuring team productivity.