Measuring Agility Without Counting Story Points: A Team's Turn to Evidence-Based Management
A software team at a services company believed it had agility figured out. They held all the Scrum events, the board was tidy, and velocity had crept up quarter over quarter. When a new director asked a blunt question — "Are we actually getting better at delivering value, or just busier?" — the room went quiet. They could report how many story points they completed. They could not say whether customers were better off. That gap is exactly what Evidence-Based Management, a framework from scrum.org, is built to close.
The timing made the question sharper. It was early 2021, the team had been remote for a year, and demand for their product had shifted as customers adapted to a hybrid world. Output was steady, but no one could tell whether that output still mattered to the people paying for it. Velocity, it turned out, was measuring the team's busyness, not its results.
What Evidence-Based Management measures
Evidence-Based Management (EBM) helps organizations improve by measuring outcomes rather than activity, and using those measures to run small experiments toward goals. It organizes measures into four areas, called Key Value Areas. The team in our scenario had been living almost entirely in the bottom-left corner and ignoring the rest.
Current Value — the value the product delivers to customers and users right now.
Unrealized Value — the future value that could be captured if more needs were met.
Time to Market — how quickly the team can deliver new capability or learning.
Ability to Innovate — how much capacity the team has to deliver new value rather than fighting old problems.
What the team did differently
They did not throw away velocity; they put it in its place and added measures that spoke to outcomes. The shift was less about new dashboards and more about asking better questions and being willing to hear uncomfortable answers.
Started from a value goal. Instead of "increase velocity," they set a goal tied to Current Value: raise the share of active customers who completed a core task each week.
Measured Time to Market honestly. They timed how long an idea took to reach customers, and found that approvals — not coding — ate most of the calendar.
Looked at Ability to Innovate. They estimated how much effort went to maintaining old features versus building new value, and were unsettled by how little was left for the new.
Ran experiments, not edicts. Each Sprint tested one small change against a measure, so they learned what actually moved the outcome instead of arguing about it.
What changed, and why it stuck
Within two quarters the conversation in reviews changed. Stakeholders stopped asking how many points were done and started asking whether the value goal was moving. The team cut a slow approval step that EBM had exposed, which improved Time to Market more than any coding change could have. And because every decision was now tied to a measured outcome, debates got shorter — the data, not the loudest voice, settled them.
The deeper lesson is that agility is not a set of ceremonies you perform; it is the demonstrated ability to deliver value and to improve at delivering it. EBM does not hand you the answers. It gives you honest measures and a habit of experimenting, which together turn "we feel agile" into "here is the evidence."
If your teams are busy but unsure whether they are delivering value, XNM's program & project delivery advisory can help you put evidence behind your agility.