How to Use Agile Metrics to Measure Your Team and Organization's Agility

Clayton Lengel‑Zigich: Welcome to another episode of the Agile Weekly Podcast, I’m Clayton Lengel‑Zigich.

Derek Neighbors: I’m Derek Neighbors.

Roy vandeWater: I’m Roy vandeWater.

Jade Meskill: I’m Jade Meskill.

[laughter]

What Metrics Can I Use To Measure How Agile I Am

Clayton: We almost forgot you are here. I want to do Agile, and I want my teams to be awesome, and go faster and be better, and do more better. What metrics can I use to measure how good I am at Agile?

Jade: K‑Locks.

Roy: Velocity.

Derek: Velocity.

Roy: I already used that one.

Derek: For me, this is so difficult because in other question I always ask, “What is your company doing?” What are the goals of your company? If you’re doing the Agile, you’re succeeding at things you weren’t succeeding at before in your company, whether that’s making your customers happier, whether that’s achieving some new functionality, or gaining more users or doing something, moving in the market.

But, to me, unless you are a software consultancy that is getting paid to be more agile, why on earth would use any metrics of agility to measure whether you are successful? Being more agile has no purpose if you’re just more agile‑like.

If I’m a thousand percent more agile, but I don’t know what the hell to build with that new agility, what’s the point of being more agile?

Jade: We are talking about problems for developers here. I’ve got to get these developers in shape. Those are problems that I need to worry about. Everything’s cool there. We’re good. Don’t worry about that stuff. The developers are what’s holding us back.

How Do I Know If My Teams Are Agile

Derek: One of the things I’ve started to do a little more, at least at the team level is ask that the direct manager for the teams to say, “How would you know that these teams are better? What are some things that you would know?”

I, basically, right off the bat say, “If your answer is more, better, faster, I’m not interested in helping you.” What we generally started to see are things like, “We want the team to be interacting, more co‑creation to happen. We want it to be easier to on‑board new team members. We want the quality of the code or the technical debt to start to recede. We want more automation. We want to be able to deploy more often.”

There’s still shallow, technical things, but they’re not velocity, or story points, or that we’re doing process X or process Y. There’s something more tangible, like, “Hey, it used to take us three days to deploy the software, now it takes us a day. We’re getting better at being able to do the deployment of the software.”

To me, that’s something, and that’s better than velocity. But to me, great, if you can deploy every 30 seconds, but you don’t have anything worth deploying, what’s the point?

Clayton: I had a manager tell me recently that they wanted to do agile, because they wanted people to stop leaving.

Derek: I’ve seen that in a couple of the value sessions, or goal sessions that I’ve done with managers, where some of it is they identify, they want a culture that is more friendly. I’ve got one team that overtime, lot of overtime, and big pushes for major events are a problem. One of the things they really wanted was more of a sustainable pace.

The reason why is because they said, “We want to stop burning people out because we’re losing good people to it.” I think those are good and noble things from a technical and team perspective. If you’ve got this great sustainable pace, but you’re crappy in the marketplace, you’re not going to be sustainable as a company.

The Only Thing That Matters Is Customer Delight

Jade: That’s why Steven Denning says, “The only thing that matters is customer’s delight.”

Clayton: If I’m kind of some average manager guy, and maybe I read “Lean Startup,” the takeaway for me is you need data and experiments, the scientific kind of thing. Then I talk to kind of the run of the mill agilist, and they say, “Metrics are kind of wishy‑washy, you don’t really need those.” Do you think that’s some of it? Where people think that they need to measure something, and science is so important. I just need and data crunch numbers and have a yes or no binary answer?

Jade: Look at the success of Frederick Taylor and scientific management. It’s build wonderful companies where people want to work…not.

Derek: I think what happens in organizations, they want “organizational agility.” Organizational agility means everybody is “agile.” If I’m the person that’s, basically, championing that, I have to be able to tell the CEO, “Hey we’re agile now!” He’s going to go, “How do you know that?” “Because all the teams are agile!” “How do you they’re all agile?” Because of…

Jade: We’re all doing Scrum!

How To Measure Organization Agility

Derek: …they’ve all doubled their velocity. I think there’s a lot of that going on. People are trying to measure that binary bit, are we agile or not? Opposed to organizational agility being about, “We’re more competitive in the marketplace and we’re making our customer happy. We’ve got better employees, and we are learning more,” and all of these kind of principles and values behind that door, like nobody wants to top us on.

I am all‑for the majoring stuff. It’s just be careful what you major, because we are going to give what you major, and people are going to focus on what you major. If you focus on velocity you might get some really great stories “done.” But the quality can suck ass, the automation could be horrible, and your customers could be miserable.

You have to be a little bit more holistic in what it is that you are really trying to achieve, and so when I always try to do is say, “Why do you want to be agile”? More, better, faster is not a thing. If you want more, better, faster, give me a whip, and let me fire anybody who does not code up to standard right‑now‑today. I’ll beat the crap out of these people, and you’ll get the best code you’ve ever seen for twelve months, from a speed perspective. The problem is that’s not what people really want.

Clayton: You might be in the checklist for agile, but you are delivering something crappier or whatever. Even from speed, I think a lot of people, ourselves included, would say that releasing frequently, and giving feedback is a good thing, so isn’t that more, better, faster? If I release every four week, or something, or at the end of every sprint?

Jade: I think it comes down to what your intentions are behind doing that right? If you are doing that to make more money, or hit these quarterly deadlines that are being set for you, then that’s not the right thing. If you are doing it because you find that giving your customers more frequent access to new updates to your software provides better feedback that allows you to build a better product. Now you are actually doing more, better, faster, but you are doing it for the right reasons.

Derek: I would say the big push off is, if I just want people to release faster, for the sake of releasing faster, that’s shallow crap, and nobody should do it. If people are releasing faster so that they can get feedback more quickly, and they can learn, and they believe they are getting feedback from a real experience instead of a stimulated experience, is a more valuable learning loop, and that that learning loop can be closed faster and faster allowing them to become more and more nimble.

Then, I don’t think that it’s about going faster, that’s about learning faster. I think that’s totally different. If we say we want faster production, then we are going down the wrong train. If we say we want to learn faster, we want to be more quickly attuned to what our customer needs, that speaks more towards agility. You might not have to even release software to do that. You could do paper prototypes and customer surveys, and you can go out and talk to customers and something else.

But you might say the problem with that is, “That’s not a real experience and I want to give somebody the real thing.” In order to do that, if I can’t produce quicker, it gives me tighter loops. I think if you are going faster just so you can say, “We are a fast team,” you are doing it wrong. If you are saying, “We want to go faster so that we can validate our hypothesis much more quickly, and respond quicker to customer demand to make them happier,” I think you start to win.

Why Is Releasing So Difficult

Clayton: What is it that makes releasing so hard? Why is it that so many companies seem to have really hard time releasing software frequently?

Derek: Lack of automation. I think that is the number one thing. They do it so infrequently. They do not know how to do it. The other thing is technical, that they are scared to release something, because if something goes wrong, it is really difficult to undo that thing.

Roy: Organizationally, too, a lot of times sales tunes are set up to deal with much larger releases. They do an annual release or whatever, after spending all this time, wrapping up and selling the big new feature, because customers purchase features.

If you are really thinking of service, it’s really obvious to do version increase, just because instead of as your customers they keep subscribing to your service. But if you sell a product, and especially when you want to charge a lot of money for the project, like a lot of companies do, like Adobe, or any other companies.

You purchase Photoshop, and it’s really expensive, so there it has to have a lot of features in there. The inverse business model to release a product like that, does it have to reselling a cheap product really frequently, in order to make some of that money back and that they probably end up making less. You know what I mean?

A Lack Of Automation Points To Lack of Technical Practices

Clayton: Derek, you’ve mentioned automation, and the pressures of the sales people wanting to sell the big features that you’re able to upgrade kind of thing. Why hasn’t there a focus on the technical practices that a lot of agile people are talking about? When you get back to technical practices, developers have lost their way, and they don’t value that stuff anymore.

Derek: For the automation side?

Clayton: Sure, yeah.

Derek: I think for the automation side, the problem is that people tend to get told they have to go really, really fast, and the pressure is on to ship features, so they don’t ever take the time to slow down to go fast.

If I got to do something a hundred times, and it would take me an hour to automate it or it would take me 10 minutes to do it, which one is better? If somebody’s holding a bat to my head and say, “I need this right now.” I’m going to take the 10‑minute option almost every single time, because I don’t want to get hit in the head with a bat if I take an hour to do it even if it means it’s free every time after this.

I think people are trained to not automate because it takes time to automate. I think a lot of the people in the industry right now are coming from environments, where they’re not very Unix based. They’re not used to toolsets and tool chains that allow for really good quality automation of passing thing from one thing to another, controlling things via command line, and a number of things.

I think some of it is technology has gotten better. It’s, basically, hidden some of what’s underneath the hood to a lot of the current developers that are out there. They don’t even know it’s a possibility to do some automation.

Infrequent Releasing A Sign Of Bad Product Management

Clayton: How much of not releasing frequently is just bad product management? Like they don’t have anything to release, or they’re afraid to release something, because it’s not good enough, or there’s no innovation.

Would you think a company might release more frequently, or they would push that issue if there was a ton of innovation going on, and they have all these awesome ideas, and all these awesome stuff that they could execute on?

Jade: I think a lot of it is overcoming the legacy of packaged software. The Internet has radically transformed our ability to release often. That didn’t use to be a thing that existed. You go back 15 years, you couldn’t do that.

Derek: It takes a couple of weeks to produce an ISO.

Jade: Yeah, at least, and it costs a lot of money. The upgrade costs were very painful. A lot of people who find themselves in product management situations, they’re still tied to that legacy of thinking that way.

Some of the Web startups, they embrace that right away, because it’s totally possible. They have come to the realization that that world exists, and they can do whatever they want. They can release whenever they want, and their customers won’t even really know. But that’s a relatively new thing that has happened.

Organizational Debt And Project Hell

Derek: One of the things I’ve seen a ton of happen, and I think this is to me the equivalent of organization, if there’s technical debt, I would call some of these organizational debt. Especially in enterprises one of the things that happens is everything becomes a project. Then all projects become interdependent. Then we can never release because we’re always waiting.

The example I used with somebody today from a metaphor perspective is imagine if you get off the airplane, you go down to catch the shuttle to go get your rental car. The guy pulls up, and you get in, and he sits there for four hours.

You go, “Aren’t you going to take me to the car?” He’s like, “Well, look there’s other people coming off the plane. Until everybody gets off of every plane that comes in tonight, we’re not leaving.” You would be pissed off.

What happens today is you get on that. Two seconds later the door shut, and he takes off. If somebody is running behind the thing and you say, “Hey, there’s somebody waiting,” his response is going to be, “That’s OK. There’s another shuttle in two minutes.”

What happens in the enterprise world because they’re not having these frequent releases and because everything is interdependent, nobody can leave to go get their car until every plane has landed, and everybody is on the shuttle. I think that…

Jade: Forever [laughs] .

Derek: Forever. It just gets to the point where a six‑week contingent becomes a 10‑week contingent becomes a 12‑week contingent because now everybody is scared shitless to release it, because they don’t even remember what’s in it anymore.

Then people starts to say, “We have to release now. We can’t wait anymore. There’s some big ad coming out. We have to do it.” Then people start tearing out half‑done features. Then they’re really nervous, because now they don’t know what they’ve added, and what they’ve removed.

I think this is one of the things that automation and that doing it regularly. If you can get to the point where you say, “Just go ahead and put it into production, the next shuttle is coming in two minutes,” it really changes how you think.

But I think it also requires that people stops thinking in terms of projects from a product management perspective, and think much more in terms of features. “This feature is done, lock and loaded. Go,” and not, “There’s this 60 features, and they all have to be released at the same time.”

Jade: It’s being tied to the legacy of selling versions.

Long Release Cycles, Who Moved My Cheese

Clayton: You made a point about this a couple of weeks ago, Derek, that you might be wanting to shift more away from that big versions or the release once a year thing. There are some people that might say, “Hey, give me every single update. I want to be bleeding edge,” and there are some existing customers that might say, “You know what, I’m fine with the one big year update.”

For the most part, that would be pretty seamless for them. They’re going to get essentially all the little updates all at once, and to them, it’s going to seem like the same old big release. But the other people, they can just get the new stuff every month or whatever it is.

I thought that was an interesting point, and a good way to think about transitioning from people who are in that, “I have to make a DVD, and mail it to people” mindset to more of the software’s a service but not jumping full shift into that. I think we’re out of time. Thanks guys.

Derek: Thank you.