The Agile Manifesto and Embedded TDD with James Grenning

Derek Neighbors: Hello and welcome to another episode of Agile Weekly Podcast. I’m Derek Neighbors. Today I’ve got a guest, original signatory of the Agile Manifesto James Grenning with us.

How is it going James?

James Grenning: Good Derek! How are you?

History Of Snowbird Meeting

Derek: Good! You know one of the things that always intrigues me when I get the opportunity to talk with someone who’s kind of been with Agile since the beginning is how maybe you see the Agile community, Agile practices.

What do they look like in back in early 2000, 2001 years, a decade later where they are today? Where there were similarities? Where there were differences? Where did things maybe worked out of how you guys intended? Where maybe there some frustrations were you’re pulling your hair out going, that it has been ten years and we still haven’t get this part of it right?

James: Yeah, OK. That’s an interesting thing to talk about. Ten years ago, or actually twelve or thirteen years ago now, when we started to get involved with extreme programming, if the time of the Agile manifesto, I was part of the extreme programming contingent community there with Bob Martin and Cat back in Ron Jeffrey’s.

I don’t mind at all saying that when I went to it it was like, “It’s in Snowbird? Of course I’m going to go.” That should be a good time. All these good people and then some skiing to boot. At the time we were just trying to see what we had in common. It was an interesting thing, we didn’t have any expectations that anyone would care.

Bob Martin, and I believe Alistair, got the whole thing started. Let’s get together and talk about what we have in common. That’s what the Agile manifesto came from. One of the things that was surprising to me at the time, coming through the 90s, a lot of people might not recognize this, but right or wrong, the way we viewed process released kind of the camp that I had been involved in in the 80s and 90s was if we could come up with a better process then the people wouldn’t matter so much.

I think that that was, that permeated a lot in the 90s, and what was really different about the people that I was with at the Agile manifesto meeting was that really they were talking first and foremost about people, and that was kind of surprising because of the Watts Humphreys message sounded like, and the way industry interpreted was if you had a better process you could just plug any person into it. Plug a replaceable programming unit and get the same results. That is one very different thing I saw about Agile. You were wondering how it might relate to today, right?

Derek: Yes, absolutely.

The Technical Practices Didn’t Get Widely Adopted

James: I think the engineering practices and such behind extreme programming were what were intriguing to me and to Bob Martin, at Object Mentor. We were really seeing that those are kind of revolutionary. It is really going to help solve a lot of interesting problems. As naive coaches in the early ‘2000s, we thought people would hear them and be enthusiastic about them and what I am still surprised at is that how much resistance some of these what we view as common sense approaches to developing software seem to be still so radical to people.

I think it is pretty interesting and cool that Scrum has taken off so big but it is fairly empty. In some of the engineering practices they have done a lot to spread the idea of iterating but they haven’t spread the idea so much of getting the quality up and really relying on high quality to go fast.

I know that is Schwaber’s and Sutherland’s intention in Scrum but it is not really the way the industry is going or has gone. That is disappointing. Although it makes lots of opportunities for guy when companies finally realize that they need that because that is the sweet spot that I want to get involved in.

Derek: I definitely think it is interesting that I think one of the beautiful pieces that Scrum has allowed is some of the concepts of agility to permeate outside of software development. Where if maybe we just had solely XP, we wouldn’t be able to bring some of the concepts of that is really all about people and working together and collaborating and iterating.

Maybe those wouldn’t make into non‑software pieces but I definitely think that one of the downsides has been there are a lot of people picking up what they really believe is Agile in software development and totally missing the whole essence of technical excellence and all of the pieces that are around that.

They think that they can be successful with just half of it and I think that anybody that has been around the community for a while understands that if you don’t have good core technical competencies and technical excellence that is really hard to iterate fast.

Being Humane And Good To People

James: You said half and I would say maybe they’re only getting a third of it. I bet we can come up with quarters next. The third is the mechanics of the interim cycle. There are 150,000 “certified” Scrum masters out there that know the mechanics of a day one Scrum.

The other third is being humane, being good to people. This is a thing that was shocking to me in the early 2000s. You just said the better process so people wouldn’t matter thinking, and the people really do matter.

The quality really matters and for that you need to really have sound engineering and people they’re not just programming as a job, but programming as a passion and an art form and a discipline, right.

Derek: Yeah. I think one of the problems that we’ve had is that for the most part a lot of people are still selling Agile as just a better process.

The people that are paying to have it adopted in their company are really taking the very ’90s approach of, “OK, if I spend money and I go get this Agile coach or I get a Scrum master, I implement this thing that I’m going to be able to plug anybody into it and they’re going to go faster and it’s just a better process.”

James: Yeah. The engineer in me still wants to behave that way. If I just show them how these good practices work then they’re going to want to do them and unfortunately it doesn’t work that way.

History Of Planning Poker

Derek: Yeah, absolutely. It’s part of that. I think that we’re seeing a lot of influences from other communities as well, certainly Lean and Kanban and other ways of…People are struggling now to hack processes.

They understand that maybe there’s some methodology, maybe there’s some principles. How can they maybe create or roll their own or experiment with different things?

One of the things I’ve seen that’s been pretty popular lately, certainly I’m a fairly big fan of Planning Poker or planning exercises to be able to understand where we might sit a few weeks out and have some decision making process before we go and spend money.

I know that even Ron and Chet to a certain degree, certainly those in the Kanban camp are leaning more away from traditional, estimating upfront and more taking the approach of “Let’s look at the work we’re doing, measure some of that work we’re doing and then see if we can get predictive capabilities based off of work already done.

As some who is maybe the godfather of Planning Poker, what are some of your thoughts about planning now several years later?

James: Well, I won’t disagree with any of the things that you just said there. The idea of thinking that at the beginning of a project we can really lay out a detailed plan that’s going to work is, I’m more and more convinced that that’s crazy.

How do you plan an invention? Just pounding at widgets would be one thing, but there’s really an invention being created there. Pretty much everywhere that software is being invented or whatever you’re applying Agile or Lean to. I suppose maybe fashion.

There’s a disconnect between the flavor of Lean and manufacturing in a flavor of Lean in design. Because one is you need variation in design to come up with creative ideas and then manufacturing you need to be able to do that same thing over and over again with high quality.

What about planning, you’re asking me. I actually tell people not to use Planning Poker because I feel like it’s too slow. At the time of that first meeting when I just dreamed it up out of a pragmatic need to get a meeting that was stalled going again, it really helped speed this up.

Then we discovered other things within the same year that sped us up even more like using affinity grouping and such, so that we could start to see which things are alike, which things are different. Then do a batch mode Planning Poker and piles of stories.

Yeah, we know that those estimates are really wrong and you’re just trying to come up with some guesses to how big something is, so you can know whether or not you should proceed. If you spend two days playing Planning Poker you’re wasting two days. You could do that in two hours using other techniques.

I hear what the Lean people are saying, “Why estimate at all?” But my world, in embedded systems, where there’s hardware and software that have to come together, the business is to have an idea of when things are going to happen. We can’t just live in the ideal world of what’s the most important thing to do next.

Although we do work on the most important thing to do next, we’ve got to try and create some vision of how long it’s going to take.

Derek: I like to tease and say, “We call them estimates for a reason. If they were actuals, we would call them actuals.” But I think I take a very similarly approach in that, really, it’s about estimating as quick as humanly possible and getting something that is accurate, not necessarily as precise as possible.

If we know, going in, that it is an estimate, and that the bigger amount of time we’re trying to measure, the more inaccurate we’re going to be, but we like to use if you’re taking more than a minute per story to come up with an estimate, you’re probably taking more time than it’s worth, unless you really need to be precise.

James: Yeah, unless you’re about to work on it. If you’re about to work on it, OK. But, if you’re just trying to come up with a release plan, for instance, that stretches out to see how insane we are for trying to do what we’re trying to do. It should be very fast.

Embedded Test Driven Design

Derek: Yeah, it’s the litmus test. Is this something we could even remotely fathom to do in this amount of time? If the answer is no, we just keep chunking it down until when we get something that’s won’t be right, but at least it will be somewhere in the neighbor of the ballpark.

I find that most product managers, that’s good enough for them. They don’t want to be told, “You’re going to get everything,” and then you don’t get anything, so, if somebody can chunk it down to a reasonable piece they can either go out and ask for more money, move dates, or do some things upfront, which removes some of the anxiety from them, really, and gives them the ability to say “Is this worth doing?”

I think you talked a little bit about embedded. I think some of the work you’re doing with embedded TDD is pretty fascinating in the sense of, I think, 10 years ago, I don’t think a whole lot of people would have thought that Agility was a place where hardware and software would necessarily all intersect with each other, but I think it’s becoming more and more commonplace in the manufactured or embedded world, especially if move to mobile devices, a number of other things.

What are some kind of trends you’re seeing in the embedded world and adoption of maybe some of the XP practices?

James: Actually, the funny thing about embedded software is, yes, there are some different things about it, but it really doesn’t matter, because all the techniques and principles…for instance, the solid design principles and having rapid feedback, these are all things that are going to be helpful, whether you’re programming on a microcontroller, an Android phone, or an IBM mainframe, if there is such a thing anymore.

The underlying principles are the same. I had this nice advantage of growing up in the embedded world and then spending a bunch of time not in it, and, when I first ran into extreme programming, it just seemed to solve some problems. For instance, one of the challenges of the embedded engineer is that they don’t have hardware usually.

What that used to mean to us is that we would code like crazy with no way of knowing if it works. Then, when we finally got the hardware, really close to the deadline, then we’d have to figure out if it works, and, of course, it didn’t. We’d get these high hopes in all of our documents, and everything we’re going to make it so that we just plug it in and it works, but, no, you had to go back and make sure everyone coded the right thing.

We’re just pretty much back to verifying each line of code, which, when someone objects to TDD, they’re objecting, “You have to verify each line of code?” Yeah, but you do anyway…

[laughter]

James: …so don’t pretend like this is different. I am seeing that there’s more interest in applying TDD in embedded, and it’s not just the TDD part, it’s the iterative cycles, it’s the breaking work into vertical slices. All these things are foreign. Engineers are famous for being chopped into their silo and shoved in a cube, and come out several months later and try to integrate something in the…

Some people are trying to change that. I’ve been working and trying to change that for the last…if I go back and look at history of going and speaking at the embedded systems conferences, it’s probably 11 years now, I started to try to get people thinking about this. I’m going next week, as a matter of fact.

Challenges Between Embedded and Non-Embedded Software Teams

Derek: Do you see a lot of the challenges as being fundamentally the same between embedded teams and non‑embedded teams adopting XP, or are there some challenges that tend to be a little bit different? Typical software teams don’t struggle as much with or they don’t even have the problem where embedded teams maybe do they’re a tool setter, the other outline factors that don’t exist for most teams.

James: If I’m a Java programmer, and I’m building a program for a computer that the Java compiler and virtual machine run on, I’m building and testing on the machine that I work on. A fundamental difference from that is you’re building a PC, a Linux box, or a Mac, and you’re running your code on something else, so there’s a fundamental difference here.

For instance, C is supposed to be portable, so why not write your code in as much as is possible to be portable, so that you could run unit tests and such off the target and then run some of them on the targets. In embedded, they called it the target system, the different processor.

But there’s a fundamental difference here. But then, when I think about the techniques, an embedded engineer might say, “I’m special, because I have to interface with a piece of device.” A business program is going to say, “I’m special, because I have a database or UI.” Now, the techniques for breaking the dependencies on devices or databases or UIs are all the same techniques.

The problem is that, oftentimes, the embedded engineer doesn’t even know they exist, because they don’t relate to software developers outside of embedded many times. I don’t want to make blanket statements, but, generally, they align more with engineering and maybe electronics, so the awareness of techniques that work and would be useful to them that come from Java, Ruby, or whatever, they’re unaware of and don’t know how to apply them.

If they could be aware of them, they could certainly improve aspects of their work. TDD is one of these examples. They wouldn’t know about it if…I don’t want to pat myself on the back too much, but I have been promoting it for over 10 years now. There are people that are starting to do it, and they have been doing it with a lot of success.

Derek: Yeah. I certainly even see that within typical software teams, where maybe you have a lot of experience, say, in the Ruby community, and then you move to a Java community, and a lot of the tools and some of the cutting‑edge things that a dynamic language brings you in some of the thought process, and certainly those that use Smalltalk and other languages going back to C.

When you get these new insights, they become second nature for you, but you don’t realize that another community that has never seen them before don’t understand some of the techniques and the principles and aren’t able to leverage them.

I think some of it is really great to be able to share cross‑discipline, to be able for a Ruby programmer to talk to a Java programmer or talk to a COBOL programmer or to talk to an embedded developer.

James: … or talk to an embedded C programmer.

Derek: Yeah, exactly. Because you can learn something from all sides. I think you’re absolutely right that they’re more similar than they are different, and we just maybe use different language about how we talk about them or how we see the problems.

James: That is exactly true. The kinds of problems people are trying to solve, and maybe the domain languages could bridge both, but the solutions base is very different as far as what things we talk about and how we talk about them in embedded versus other areas.

Derek: We’re at the end of our time box. Is there anything that you’ve got coming up that you’d like to share with people? Any books, events, training, you name it, that you’d like to share?

James: Sure. Well, you could look at my website. I got a root website, JamesGrenning.com, that’s easy to remember if you remember my name. There’s not much there, but it’s got the links to other stuff I’m involved in for instance…My business is coaching and training.

I’ve been quite busy helping embedded developers get started with TDD and deal with their very difficult and challenging legacy code problems. I’ve also got some public training coming up. You can look at my website. Usually, there would be a banner in the corner about that.

There’s one coming up in October. We haven’t scheduled the one for the spring, but we’re going to do another one in the spring with one of my partners. I’ve got the Embedded Systems Conference. If you listen to this podcast right after it was published and maybe not before the Embedded Systems Conference in Boston happens [laughs] , and you’re out there, come and see me.

Derek: Sounds great. Thank you for your time. We really enjoyed having you.

James: Hey Derek, really nice to talk to you.

Derek: All right, thanks.

James: Yup.