Is the Process the Problem?

Drew LeSueur: Hi. Welcome to another episode of the Agile Weekly Podcast. I’m Drew LeSueur.

Derek Neighbors: I’m Derek Neighbors.

Clayton Lengel‑Zigich: I’m Clayton Lengel‑Zigich.

Roy vandeWater: And I’m Roy vandeWater.

Don’t Change Process Because People Aren’t Following It

Drew: Today we’re going to be talking about an article by Jamie Flinchbaugh titled “Don’t just change the process if people aren’t following the existing one.” The article talks about how people focus on the process and following it to the tee, as opposed to questioning whether that’s a correct process. What are you guys’ thoughts?

Roy: I don’t know. It sounds like it could be very easy to abuse. It sounds like an invitation for scrumbutts, or any other process butts, “Well, it’s a problem with the process.” I guess this is, actually, saying don’t change the process. Actually, I like that. I guess maybe it’s not a scrum‑but right. It’s saying, just because it doesn’t work for you, maybe it’s not the processes fault. I completely turned around right there. [laughs]

Process Doesn’t Fix Problems, It Highlights Problems

Derek: What I see a lot of times are people get so hung up on their process as the problem. I think what he might be saying is, if you’ve got a process, and people aren’t following it well, what makes you think they’re going to follow some new process? Even if they are following the process, is how do you know the process is really the problem? Meaning, if you’ve got deep culture issues, or you’ve got leadership problems, or you’ve got visioning problems, process does not solve those.

All process does, especially if you’re talking Scrum or Agile, Kanban, all it does is highlight problems. It doesn’t actually solve them. If you just say, “Here’s process A,” and we run through it, and it highlights all these problems, and we go, “Hmm, yeah, let’s not do anything about these problems, let’s just switch to another process, and maybe if we have to have that process those problems won’t exist.” Then that process highlights the same exact problems, at some point you have to deal with the problems.

Drew: Derek, how often have you found that it actually is the process that is the problem within a company? Has that ever happened to you? Or is that something that happens generally speaking and this is an exception where it is?

Derek: I’ve never seen the process be a solution. I have seen a lack of process, or no process, make it so that a company or a team does not know what the problem is. I think sometimes you have to jump into a process to help highlight what the problems are. But, ultimately, the process never solves the problems, it just highlights them.

If you’ve already got a process that’s highlighted a bunch of the problems for you, I wouldn’t advocate going and switching to another process. I would deal with the problems that your current process is highlighting. If you have no clue what problems you need to tackle, then maybe you need some process to help highlight the problems for you.

Not Enough Time For Planning

Clayton: Yeah, I think that one example that comes to mind is you might find a team that is doing Scrum, and they never have enough time for planning. Like, every time they say, “Scrum, the process says we have to do this thing called a planning meeting,” and it has these certain things we have to do, and there’s this artifact or whatever that falls out of it. But we just can’t just do it.

Either the work we’re doing is too complex and we don’t have the time to break down all the work that we’re doing in this planning meeting, or there’s just not that much time in a day. They told us we have to do this two week sprint thing, so we can’t spend two days. That’s too big of a percentage of our Sprint just doing planning and thing that’s one of those situations that Derek’s getting into is, “OK, well, maybe there is a problem. Why does it take so long to plan?” Or maybe that’s not even a problem. Maybe that’s OK. But I think that’s the kind of thing that people will dive in and say. “This process is broken, it doesn’t work!”

Roy: Does that mean that maybe processes like Scrum, and like some of the other ones out there are really just a way to normalize your team, so that you can talk in the same language as all the other teams? For example, having the issue of not being able to spend the time for planning, you might be able to go to another team that is practicing Scrum, and say, “Hey, we’re having this problem, and because you guys are using the same terminology, and you are trying to implement the same process, maybe all it is, is a framework to make it easier to communicate your problems to other teams, so that they are able to help you out.”

Clayton: Yeah, that probably would make it easier, but I don’t think that’s really the goal so much as the teams have probably going to all have some different kinds of problems. They are all going to be exposed in some different way. I think it’s kind of, you get to that fork in the road, where either you decide to, “OK, well, here’s this problem we have. We’re going to resolve it somehow.”

I think what the gist of article is, “We have this process problem. It must mean that we’re not doing something about the process right so let’s stop doing this and find a new process.” That’s the real issue. A lot of times, the stuff that something like Scrum or any agile methodology will uncover are difficult things that you would rather not have to solve. It’s a lot easier to blame it on the process, do some new thing, “Here, look, look, a pony.” You get some new thing in there, and you can kind of start the cycle over again.

Negative Processes

Drew: We have some processes out there that I feel like most of us consider negative processes. I think, in most circumstances, for example, waterfall is pretty much frowned upon in the agile community. There probably is a time and a place for it, but how do you make that determination?

I suggest if the entire agile community shuns it, then we can’t consider it a valid process. It might be the process that is a problem. How do we know that it is my process that is a problem, or how do I know it is my team that is a problem? Do you know what I mean?

Derek: I think a process lets you know what challenges you face within a team or an organization. I think the reason that waterfall has been universally shunned over or shown to be not so effective is it’s feedback cycle is way too short. It does have the ability to tell you that things are wrong, and to deal with them. However, the time frames that that are in are generally in months or years as opposed to days or weeks.

I think when I look at why most agile processes do fairly well is because they’re very iterative, which means they’ve got much, much smaller feedback loops. I would almost argue that a lot of teams I see doing scrum are really doing mini‑waterfall.

Their teams are still very siloed. They’re doing two or four‑week sprints. In whole, they have iteration zero, and they have a hardening sprint. They have all these smells, but they are getting feedback in a much tighter loop than if they said, “Hey, we’re going to do this 12‑month project, and we’re not really going to do a postmortem until month 12.”

Clayton: Just to clarify, I think you meant to say that they have a longer feedback cycle?

Derek: Yes, longer feedback cycle. I’m sorry.

Resistance To Process

Drew: When people have resistance to part of the process ‑‑ like, “Oh, we can’t spend all this time planning” or “This standup is just getting in our way. Why don’t I just get my work done?” ‑‑ part of a lot of agile processes are ceremonies that may be uncomfortable for people who are trying them to start.

What are ways you guys think to overcome those type of awkwardness of starting a new ceremony that, by their culture, they’ve never done before?

Roy: I think a lot of it is providing value in those ceremonies. I see way too many organizations institute a standup or a planning meeting. They are pretty much dog and pony shows in the sense of, they are there only because some scrum book or some scrum master or some coach told the team they need to do that, but the teams get no inherent value out of them.

I think the key is getting the information out in the ceremony, dealing with it, reflecting on it, and improving the ceremony every time. Otherwise, what’s the point? There’s been several times I’ve told teams, “Why do you even bother having a stand up, because what you’re doing is not valuable to yourselves or anyone around you?”

When Should You Change The Process

Clayton: Going back to the article about, “Don’t just change the process,” when do you guys think it’s OK to change a process? How far does it matter to team maturity?

Or if you get to try it for a certain period of time, how would you know now is the right time to change things and do something different even just as an experiment? How would you know how far to go with that? If you keep that for a longer period of time, is it OK to change process? How would you know that?

Drew: One example that I have ‑‑ it’s just maybe a change of not necessarily a process but a practice ‑‑ is we were trying to get our stories smaller. We had a problem of having stories that were too big. I remember during planning once, we argued forever or discussed for a long time, “Should the stories be broken up or not?” At that point, we decided, “OK. Let’s just put it into this sprint. Let’s not talk about it more. We can move on and figure out if it was too big or not.”

But I think sometimes, because of a time constraint, you can just move on, and, maybe, just skip some of those things that, maybe, you know you should do, but as a team, you don’t have consensus.

Derek: I think it’s all about data. I would ask, “Why did the team think they needed to break the story down to a smaller story?” What behavior or what symptom or what data made you think, “Hey, we should potentially break this down”?

Drew: In that experience, I think it was more of just people saying, “Hey, we should break this story” or people saying, “Big stories are bad, and smaller stories are good.” That was all the data that we were going off of.

Later on, we actually learned that ourselves when we learn that, “Hey, it’s getting near the end of the sprint, and we have no story signed off. How can we fix that problem?”

Using Data To Better Guide Change

Derek: To me, it’s all about data. If you say, “Hey the problem is the data that’s coming up is that we’re not getting stories into pending until the end of the sprint, we should change something.” Once you change something, you should then be able to measure, “Once we did that change did the problem of not getting stories into pending until the end of the sprint go away?”

If the answer’s yes, then that was probably a good process change, or a potentially good process change. If the answer’s no, then you say, “Hey, that’s not had anything to do with it, let’s either go back to what we were doing before or let’s find what the real problem is.”

Roy: It sounds like in that type of situation you’d want to be very careful to make only one change so that you know which change caused your measurable effect. Doing a process swap, like you had asked when is it appropriate. It sounds like doing a whole process swap is very difficult, because sure you can measure whether or not there was a successful change, but you don’t know which aspects of your new process made that change.

Chances are, if it did improve, then your ideal process as a team lies somewhere between the old process and the new process. In some cases, not all cases, obviously.

I kind of feel like instead of approaching a new process, what I would do is try to get the existing process working, so that you know that you are following all of the ceremonies. Deal with those problems that arise. If you still are noticing problems within the organization, or within the team, I would start pulling in one aspect at a time of the other process and trying them out. Trying to see if, just swap it in place, and see if that works for your team or not. I can’t really think of too many cases in which different best practices from multiple processes are mutually exclusive.

Clayton: I was going to say, what if the different aspects that you might be pulling in are conflicting?

Roy: Since you’re only pulling in one aspect my assumption, and it could be a wrong assumption, is that it should really only affect one aspect of your existing process. I still feel like that’s a one change that you’re making.

Derek: Maybe the way I would say it is if your current process is not highlighting problems for you, by all means, change your process. If your current process is highlighting problems for you, until you’ve dealt with those problems, changing your process will not help you.

Roy: If you think you kick ass, you’re using the wrong process and you need to switch immediately.

Derek: Maybe.

Roy: I wasn’t being totally facetious. Let’s face it, no team is perfect. Everybody needs some proof. If your process is telling you that you are perfect, something’s wrong.

Clayton: Yeah, I guess that’s a matter of does your process support something like continuous improvement, and those kind of things, right?

Derek: Correct.

Getting Process Dumped On You

Clayton: I guess I have one last question. If I’m in a situation where maybe I’m part of a team, and all of a sudden some ‑‑ let’s say scrum ‑‑ scrum‑thing gets dumped on me, and I’m told now you do this process.

We used to do this thing over here, that’s the old and busted, now we’re going to do the new hotness. Do it. Is that kind of culture, and that kind of thing even conducive to having that process be successful in the first place, or is that the kind of culture where people are going to blame the process for problems?

Roy: That’s a good point. When something like that is mandated from on high it makes it very difficult. I think we’ve had some good arguments in the past about the power of self‑organizing teams, and I think if you start organizing the team for them, it’s going to be difficult to get buy‑in.

Self-Direction vs Self-Organization

Derek: I mean, I think it’s the difference between self‑direction and self‑organization. I think that you can still have a large amount of self‑organization and be put in a container. You could be put on a team and say, “Hey you have to ship product x.” Technically, that’s not self‑direction, but you can be pretty self‑organizing within that.

I would say that, if you’ve got leadership, or you’ve got somebody who understands self‑organization, who really wants the team to quickly adopt, or quickly accelerate through a new process that is highly self‑organizing, I think you’d probably want to go through some form of a process, where you start to talk about the old process, or lack of a process, and say, “Hey, doesn’t it bother anybody else that we can’t do A, or we can’t do B, or that this is a problem, like our quality sucks, or we don’t have predictability,” or these kind of things.

Start with isn’t this a problem. If the team starts to identify it’s a problem, then you can kind of say, “Well, what if we use this thing,” whatever the process is to try to help solve some of those problems. Then, now it’s kind of the team is kind of pulling it along. It’s not you will use this. It’s rather, “Hey, here are the problems we’re trying to solve. Does anybody know how to solve them?” I’ve heard these particular processes might be good for it. Now you’re kind of letting them be part of that process of choosing the process.

Roy: The best part is they might actually have better ideas than ones you’re already aware of to deal with some of those issues. They may be able to point out that what you think are the issues are not the real issues.

Drew: They can adopt the changes, as you talked about, Derek, if they show the problems or if they share with the problems, then they can choose some change that they can make to solve those problems, and make it more of a gradual thing.

Roy: It’s their victory if they succeed.

Derek: I think it falls in line with, if we’re talking at least scrum. It falls in line with you can tell the team this is the story that needs to be done, but you’re not telling them how to implement that story. In the same way, you can say, these are the problems I need as a product owner, or these are the problems I need as a CEO, but you’re not telling them this is the process you have to use to give me that information or to fix those problems.

Drew: Thank you. That concludes our episode. We’d love for you guys to join the conversation at facebook.com/agileweekly. Thanks a lot.

Clayton: Thank you.

Derek: Bye‑bye.