Why Enterprises Crave Documentation Artifacts

Chris Coneybeer: Hello again. Welcome to another addition of ScrumCast. I’m Chris Coneybeer.

Roy vandeWater: I’m Roy vandeWater.

Derek Neighbors: I’m Derek Neighbors.

Drew LeSueur: I’m Drew LeSueur.

Why People Crave Documentation Artifacts

Chris: Today, trying to go through topics and something we thought about was large corporations and the need for artifacts during sprints and during projects using agile methods. I’ve worked in several large corporations where have been beaten under with pages and pages of requirements, tracking, and all kinds of other documents or artifacts needed for managing a project.

I want to get the group’s feedback on what your feelings are and also with some of the people here that have been working offsite at other teams. Have you come across these issues? How have you tried to deal with that in adopting agile and scrum techniques?

Roy: I think part of it with the documents ‑‑ what is the real value that they are trying to get out of these documents? What are they trying to accomplish by having these documents as part of their process?

Chris: I think traditionally what I’ve seen with a documents is that they have been put in place to try to enforce processes, when other ways weren’t working for making sure that these processes were happening ‑‑ such as release management, and making sure the security have happened, code checks have happened, just sign off.

People weren’t doing their jobs. People weren’t being held accountable, and they weren’t doing what they said they were going to do. Documentation and processes were put in place to try to clean that up.

Documents For Accountability

Roy: Would you say that most of these documents or most of these artifacts are documentation of accountability?

Chris: I think accountability, but also user requirements. I’ve seen some rather large user requirement documents that if you had interactions, if you had conversations with a business owner and the stakeholders, you could do a lot better job in understanding instead of just throwing a document over the wall, and then waiting for something to come back over.

Documentation Highlights Silo’ed Organizations

Derek: It’s right, because I used to think of the desire to over document. I think we can all agree Agile is not about documentation. But I used to think it was all about cover your ass ‑‑ meaning that the reason that most of the managers I saw that were demanding documentation as a team member, I always felt that they were doing it because they didn’t trust the team.

They wanted some artifact to prove that if their boss yelled at them, they could say, “Well, look. We documented all this. The documentation has been for a long time, and nobody commented on it. How would we know?” But the other night, I was re‑reading Lisa Adkins’ “Coaching Agile Teams” book. She said something in it that made me just now think about documentation.

She said when she was a typical project manager before adopting Agile, one of the things that she said is she didn’t ever have to deal with conflict. The reason she’d never had to deal with conflict is because the way that the processes were built. Everybody was siloed. You were only on a project for a certain amount of time.

If I am the manager of a testing team, I’m only on the testing project for a certain amount of time. There are all sorts of conflict in whatever brewing up it doesn’t really matter, because in four weeks we’re not on this project any more. All the conflicts we had with other people were gone.

I think this was one of the key reasons big companies really want documentation, is they are so siloed. They play pass the baton so much, that they’re definitely afraid of it. It’s almost like having turnover every four to six weeks inside of a company. If you could think about your entire team leaving and another team picking up where you left off, that’s really scary.

The best way to combat that is why I want all sorts of documentation. When I’m the new guy coming in, I’ve got my cheat sheet to go back to if I get confused. To me, is this part of the problem with big organizations? Is it they’re not fluid enough. They’re not cross‑functional enough. It incurs the desire to have documentation to cover up for that.

Roy: I think that’s a really great point. Obviously, there’s less of a need for as extensive documentation when you’re working in a more open environment, where the team communicates verbally, primarily. I also really think the comment, or the phrase, or the idea that the main artifacts should be the code and the tests, is really powerful.

What actually is happening? The documentation can say whatever. It can become outdated. But the code and the tests that pass are, actually, real concrete documentation, and commit histories as well. I think in an environment where you’re more open, where you’re not siloed, where there’s more of verbal communication, and where you have good code and good tests then there’s a lot less need for extensive documentation that is probably not read very much, anyway.

Documentation Shouldn’t Replace Communication

Chris: The one thing I throw out there is that in regard to this documentation conversation. I don’t want people to get confused and say that we’re trying to say throw out all documentation, because there’s still going to be some documentation that’s needed. You’re going to have compliancy issues. You are going to have maybe general architectural diagrams for a high system level understanding. What I have seen, though, is that since I have been at Integrum I have learned communication.

Communication is everything and team ownership of code is everything. That makes a big difference and what I have seen in corporations is that they are replacing communication with documentation. That is where, Derek, maybe the reason some of this turnover is happening is because there is no communication. Who is happy on a team when you are not communicating with anybody? They don’t realize it and maybe that is why people will get upset at what they are doing.

How do we try to replace communication? Replace that documentation that is being used as communication that we are used to in the projects that we manage?

Roy: Part of what Derek initially stated where you had previously believed that it is a lot about trust. I think a lot of that still is true, a lot of is about trust, and that if you have a team that is consistently underperforming and whether it be because they have only been on the project for two or three weeks, because everyone is on the project only two or three weeks at a time.

If people stay on a project for longer and start performing and start doing well, I think that their managers will find less of a need to require documentation of them. Because they know that this team is going to do what they say they are going to do, regardless of whether or not they fill out the whatever report. Also as far are teams switching out really frequently, what is the primary reason for that?

Treating People Like Resources Instead of Humans

Derek: I think it is because everybody is specialized. If I am the enterprise architect, I am only needed really at the beginning of the project full‑time, and then after that you just consult me if you feel that there is a change in architecture.

If I am the QA team, and you really consider a story done when it is coded, not when it has gone through full Q and A, I am looking at the project in a much different time than everybody else. I need all sorts of artifacts to come along with me, because you are on another project. As a developer you are working on another project potentially by the time I am doing a large portion of the QA, or the enterprise architect is now on another project by the time it gets to it.

I think that there is something to be said for communication pathways. I think large companies have larger teams, which means it is harder to communicate face‑to‑face or person‑to‑person. It is easier to say we need to write that down so that anybody on the team can have access to that even when we are on the same project.

But when I think you add in the silos and not being cross‑functional, to me, it really comes down to people are treated like resources instead of like people. Managers treat everybody like a completely interchangeable widget that they are just pushing and plugging, so it is really easy to say, “I really like the work that Roy is doing and I need Roy this week, so I am going to pull him, pull that widget out and plug him into this other product.”

Now all your knowledge is gone, and because now you are on another team, it is no longer culturally acceptable for everybody on your old team to communicate with you to ask you information.

If you didn’t leave a litany of artifacts and you hadn’t been communicating with that team, all of your knowledge is now gone. The way managers try to protect themselves from that, I believe, is they want people to document absolutely everything so that they are guarded. If their resources get taken away, they’ve got a backup plan or a transitional plan to bring on new resources.

Chris: That goes so much back to silos.

Drew: Yes, I agree as well. Even if there is a situation, like you were saying, how somebody is just kind of plug and play like a widget out from one team to another, I still think that the need for extensive documentation is still not needed if the coders, if the programmers write good tests, write good code.

Sure, maybe give a shorter overview of written documentation that is not part of the code or test as needed just for kind of a big picture. But if they are doing things the right way, then yes, still extensive documentation, extensive artifact, extra artifacts are needed.

Stop Emphasizing Specialized Roles

Roy: Do you feel that it is necessary for large corporations to have all of these specialized roles? Like people who are…

Derek: No.

Roy: Especially because I understand that if we take it to a tool shed metaphor. If you have one employee who is good for everything, something that is used for everything like a hammer that you can use for whatever tool. It is great to have on hand, because you can use it wherever you go, but having a specialty tool is way more effective in those key circumstances. Does that metaphor carry over into the business world or is that a broken idea?

Derek: I don’t think it does entirely because the two things that I think that it leaves out is, one, it leaves out what you get from somebody who is well rounded meaning that when you’re really talking knowledge work, it’s not cutting dry of, “Here’s the band saw I need. Cut the band saw, and now I’m taking it over and using sandpaper, and sandpapering it.”

The work that teams do is dynamic enough that you really need to understand the whole concept of woodworking. It’s not just a matter of being able to understand like, “I’m the best drill press guy in the world.” You’re probably not making a whole lot of great furniture if you only know how to operate the drill press.

If you get to be a little more rounded, you may not be as good as the guy who only works the drill press all day, but you’re now also be able to create things that somebody else that is unifunctional is not capable of doing. I think that that’s probably the biggest piece that, to me, pulls on to the second side and that is that you’re not nimble if you have a bunch of specialist, because you’re limited by your resource allocation.

If I say I’ve got one of these guys, five of these guys, six of these guys, or eight of these guys, and all of a sudden a competitor pops up, and I need to handle something. I need to extend the software. I need to create a new project or do something, and I don’t have an enterprise architect ready right now.

Now I have to wait to when is the next time an enterprise architect’s ready. “OK, six months.” “OK, we can’t start that project for six months.” Whereas if you got well‑rounded people, it’s a lot easier to be able to pivot and to shift because even though they might not be the best in a particular thing, they’re able to get up and running with that.

If you’re in an environment where there’s communication and self‑organization, what I see is people make the right choices. If I really suck at JavaScript, and I’m going in a project that’s heavy JavaScript, I could tag out and say, “You know what, Drew, you really need to be on this project. I’m not the right person for this. We need to shift places.”

If we’re allowed to do that, the right things happen. Or, “Hey, I need you to pair with me on this, because I want to get better at JavaScript, but I don’t want to slow this project down, or I don’t have the expertise to do it.” I think if you let people make good decisions, they’ll un‑silo themselves.

Chris: You got any other comments on that?

Drew: I think that’s a great point, the pairing. It merges into cross‑functional teams discussion a little bit. But yeah, pairings pave for that.

Improving Communication

Chris: I think it really comes down to a lot of, like you said, Derek, getting the teams to communicate, being able to get people to work together, and also, not just communications on the teams but communication throughout the business. Let’s get rid of 500 page requirements documents and start the communication channels with the people that are involved with the software.

Roy: I’ve met quite a few developers that are not a big fan of doing communication. There’s a lot of people going into the computer science industry that I remember from college that couldn’t wait to graduate, get their own cubicle, and never have to deal with another human being again.

How do you feel that people like that that are hiding behind documentations and using documentations as a wall around them, so they don’t interact with people? How do you think that affects the company?

Chris: For myself, I think it impacts the company poorly because how are you going to really understand what you’re writing software for if you just surround yourself with documentation?

How many times does it help when you, actually, have a one on one talk with a user, and you see the frustration, you see their eyes light up when they think about something, or you actually watch them work? There’s something to be said about that, because at the end of the day, we’re writing software for human interaction, so we need to have human interaction to understand how that works with the software.

A piece of paper doesn’t tell me that. All a piece of paper does is signify what made it through the process that probably higher ops or whoever’s in charge of the project [inaudible 13:55] out and say, “No, this is as important as this is,” but they aren’t the people using the software at the end of the day. There’s something to be said about having that communication.

I think no matter what we’re always going to have people on the field that are going to stand behind the documentation, and we need to learn how to get them on the team, and how to use them as effective resource.

Like Derek said, make sure that they’re not just a hammer. Make sure that they’re well rounded, and we can use them in many different circumstances. But make sure that also you have components on the team that are able to have open honest communication about what are the problems that you’re trying to solve.

Communicating User Requirements

Drew: Coneybeer, I really like your answer to Roy’s question, people who wall themselves away with documentation. There’s also that documentation of user requirements where that can be minified or reduced with just communication.

Here we’re not talking about just communication between the team but communication with the end user. I really like that, because the communication in the team and out of the team reduces the need for endless writing.

Roy: I think a lot of times it unlocks a lot of innovation too where I, as a product owner, may build the requirements documents and say, “I want a piece of software that performs all of these functions and has all of these features.”

In a non‑communication sense where it’s purely documentation, I received that requirements document, and I start building the software based off of that. At the end of the day, I give them some software, and they say, “This doesn’t give me the value that I’m looking for.” The developer can say, “But look, I can check mark all of the requirements and [inaudible 15:23] those.”

But if they had taken a moment from the beginning, taken the time, and got together, and talk about what the user really wanted or what the product owner really wanted in that case, they may have come up with a way better solution that’s maybe easier for the developer to implement, that maybe gets the value to the end user better.

That would have been a better overall solution. Instead, they choose a limited form of communication to try to tell the other person what they want when they really know what they want.

Chris: Because how many times have you had that conversation were you do get a chance to talk to somebody, and all of a sudden, you both go on the same page and go, “Wow, we could do this, and it’s going to be hell of a lot cheaper, hell of a lot easier. It’s going to be a better solution”?

But too many times when people write up documentation, they put these limitations on their knowledge ‑‑ what they know how to do or what they’ve seen ‑‑ that hampers and changes the way that they’re writing out that documentation.

Then, you’re already starting with some limitations in place based upon what that person knows. When you have that communication, you’re able to figure out better solutions just like you said. How many times do we have that where it’s just like, “Oh, we could do this so much easier”?

Guy In a Room

Derek: To me, I think the question was something to that effect of, “What happens to the person that just wants to hold themselves up?” To me, this is one of the biggest detriments we’ve seen in this industry in the last 20 years.

When I look back at the ’50s and the ’60s, some of the most prominent software developers were women. You didn’t have a cube. You had a computer room that was bigger than most offices that you had to operate in.

There was a much more communal aspect to computing in the early days of computing. I think that it showed in the type of innovation, and the type of work that was being produced at that time, and the quality of the work being produced.

I think as we’ve turned into the, “Put the programmer in the dark cave, and feed him food underneath the door, and he’ll spit back code” has really hurt our industry and innovation that we see.

I think what we’re starting to see right now is, as teams start to adopt agile methodologies, they become much more diverse. They attract a much wider range of individual. You start to see their solutions has become much more creative, and you start to see the people who want to work on those teams interact in a much different way.

I think you might see a day where there’s not very many jobs available for the coder that just wants to go in, and basically bang out code, and be left alone because I don’t think that’s where innovation lies. I don’t think it lies in one person’s head, and I think history proves that to us all across the board, medicine, architecture, you name it.

It’s not one person in a dark room being inspired by what’s in their own head. It’s by being inspired through conversation and interaction with the environment. I think that’s the type of development we are moving to. It’s a more humane style of development.

Until then, see you next time.