Whenever success is called into question, there is a measuring stick that is used to define where the difference lies between success and failure. With teams, products and dare I say projects that classify themselves as Agile there is no difference. However, the biggest difference may exist in what is measured and why. Let’s address a few questions then in our journey…
What are You Measuring?
This perhaps where the discussion begins all too often. There are many things that can be measured. Completed tasks, completed user stories, defect rates or escaped defects, average cycle time, throughput, velocity, schedule or cost variance and tons of other measurements that may be classified as Key Performance Indicators (KPI’s). Knowing what you are measuring helps to at least start the discussion about data integrity and any comparative analysis over any similar periods of time. Being an accounting major, I love what numbers show. They tell a great story when data integrity can be maintained, but you have to understand what is behind the numbers themselves. Predictability, sustainability, quality control, budgeting allowances and many other things get called into question with numbers that are often tracked. What you need to measure will then lead to the next and potentially most important question.
Why are you Measuring? Why is the information important?
Perhaps the greatest question that hasn’t come from the mouth of a four year old, on a drive to Disneyland, is the question WHY. It reveals the root of why information is truly important to be tracked. Why is it that velocity is tracked for a team? One short reason is that it helps a team in a relatively short period of time (2-3 sprints) to know what they can or cannot commit to for any given sprint. That in turns helps them be more predictable for their Product Owner who can set proper expectations with stakeholders on what is being committed to and projected over the course of multiple sprints or a release. Even that being said, velocity is only valid for the team that is being measured and it is not a comparative tool across teams.
Looking at numbers like average cycle time gives us an indication of the rate or speed at which user stories are being completed. When average cycle times start to reduce below the duration of the sprint, that’s when incremental completion of committed work in a sprint is happening. My experience with countless teams has shown that when the average cycle time is less than 5 days, that’s also when teams are breaking down user stories small enough to ensure they can be completed within a sprint. It is a great measure of how well a team is working together with their Product Owner to ensure work is ready to come into the sprint and valuable enough to provide tangible outcomes with the completion of each item.
Who needs the information being tracked? Who is the information important to?
Following the WHO, WHAT and WHY model from User Stories, it should not be a surprise that we ask WHO needs the information that we are quantifying as important to share. Knowing all three of these aspects can help us know what the person is expecting. Is it the Product Owner that needs the information? Is it an executive that needs the information? Is it the team that needs to know the information? Depending on the audience consuming the measurement, we can then target that individual or groups needs to consume the information in a way that helps them determine how they can best help the team to achieve success. And yes, sometimes that means that they don’t need the information in the first place. Even well-meaning executives who ask for statistics on the success or failure of a team need to be told NO when asking about certain pieces of information. This all leads to the final question to be addressed.
When is the information needed?
Identifying the timing of sharing measurements helps provide greater context to the nature of the measurements themselves. When gives a time bound idea as to delivery of information that enables decision making opportunities. Understanding complete context
behind the use of a measurement or metric being reported gives greater visibility inside and outside a team and product area as to when something may be completed. It also gives an idea as to how to present the measurements or statistics needed to prove the level of success that is occurring with an Agile team, product or project. Having a simple dashboard that displays updated information that gets updated at the completion of every sprint could be sufficient level of information to satisfy the demands of when information is needed. Having a live dashboard that updates with the incremental completion of user stories and/or tasks is nice to have with the use of various online tooling options, but it doesn’t increase the accuracy of predictive modeling that should still be built around completion of sprint goals by teams and not rate of completion within the sprint itself.
In all, when discussing metrics or measurements of any sort there must be the questions of WHO, WHAT, WHY, WHEN and HOW addressed in order to give the best possible context and transparent delivery of information to stakeholders and others wishing to see the measurements in question to enable their decision-making abilities to be better informed. Without addressing even one of the first four questions, how the information or data is being shared back could be skewed in a way that tells the wrong story. Make sure context is understood and don’t just measure things because someone asked you to. Find out who needs it and why. Find out when it is needed and see what can be done to help those measurements coincide with the natural sprint cycle being used by the team(s) being measured.