I’ve met various peers in an agile coaching role and there seems to be no general consensus on what metrics should be used, and with whom they should be shared. I often notice a lot of confusion when it comes to this topic, so I’d like to suggest some metrics that I like to use, and call out those I find less useful.

What is a metric?

A metric is essentially a measurement that can be used to assess ‘something’. Whilst there are many things that can be assessed, when it comes to software development metrics tend to focus on performance, progress, quality and efficiency. Metrics can be technical, with the use of tools like statsd and Grafana to measure error rates and response times, but they can also be used to measure softer things such as number of solved tickets, or cycle time.

What metrics to use?

This really depends on what both your organisation and team requires. If you think of a generic software program, there are many things you might want to measure, such as its performance or usability. The same applies to agile methods, although the most common metrics I’ve seen used in the community are:

• Velocity - the number of units of work, usually story points, completed in a certain amount of time (for example, a 2 week sprint)
• Cycle time - the time the team actually spends working on a task. It is measured from when the issue enters the “in progress” (or equivalent) phase and ends when the issue meets your definition of done.
• Lead time - the time taken from when a task is logged until work is completed on that task and delivered to the client.
• Mean Time To Restore (MTTR) - the average time to recover from and restore an IT service or production failure
• Predictability Measure - how consistent a team is at producing work over time

Who are they for?

This is the most important question that needs to be answered, ideally before you start reporting and measuring. There’s zero point in collecting metrics if you have no idea who or what they’re for, or if you have no capacity to actually analyse them. The amount of detail and what is going to be done with the metrics is dependent on who you’re sharing them with. Each metric should be introduced with agreement from the team; there’s nothing worse than feeling like management are using them to spy on performance.

How often?

How often you’re sharing the metrics is also important. I like to show charts displaying the results of the metrics at every retrospective to give the team an idea of how they’re doing. By showing charts I don’t mean printing out JIRA control charts and expecting the team to make sense of it. I’m not the biggest fan of JIRA reporting, so I took Håkan Fors’ CFD template for Microsoft Exceland tweaked it until it worked for me (I’ve used this for almost 2 years now). Sure, you can argue that you can’t compare between sprints, or topics, but it’s certainly a good baseline for discussion. If the team understands and cares about the metrics, it’s easier to spot when charts look a bit more unusual and open a discussion about why this has happened.

Competitive metrics

These are the type I try to avoid. Let me explain - once you turn metrics into a competition, e.g. “which team has the quickest cycle time?” you introduce a type of culture that can make teams feel they’re not performing as well as others. For me, this is the same as equating the number of lines of code written to how good a developer is - it’s completely abhorrent. Metrics should be there to identify problems and help teams improve, not to be used in a negative way or risk the morale of teams decreasing. It’s best to compare metrics against the same team’s past performance, not against another team’s performance as they’ll likely be operating in an entirely different context.

Charts

I’ve written briefly before about burn-downs vs Cumulative Flow Diagrams. I’m still in favour of being proactive with forecasting rather than reactive when it comes to velocity. Therefore, I am still using the main metrics associated with CFDs - lead and cycle time. I mentioned in the ‘How often’ section above that I tend to avoid JIRA’s charts - I know a lot of agile coaches swear by the reports, but having started with creating burn-downs with nothing more than a Sharpie and flipchart paper I find JIRA’s implementation ‘clunky’. Instead I make use of the various gadgets on JIRA dashboards and use these as a basis for my reports. The dashboards allow me an at-a-glance view of how the current cycle time is, or indeed the most recent MTTR without having to look at a specific chart. If you and your teams find charts useful then go for it, but you don’t need a fancy looking diagram to display the metrics - as long as you know the day a ticket entered development and when the end user started to use the feature on a production environment then you can calculate your cycle time.

So should I even bother with metrics?

I would argue that metrics aren’t something you should worry about if the team are brand new to a process (whether that’s scrum, kanban or something else). Let the mindset sink in first before trying to measure how things are going. In a new team, measurements are going to look a bit strange until the dust settles and the team finds their flow. Once you’ve started delivering and the team are working together then you can start to think about what and how you should measure. If management put pressure on the team to provide metrics, the coach should be able to communicate that the team needs to ‘find their feet’ first, and try to decipher what it is that management want to know by using metrics. Once you have this, you can start thinking about what metrics would deliver this information and whether you have the data to be able to report back on this.

If you’d like to find out more about metrics, I recommend Thomas Blackburn’s presentation on Key Performance Measures in a Lean Agile Program which covers more of the principles and challenges than I’ve mentioned here.