Recently I had an opportunity to speak with Alan Atlas, a Certified Scrum Trainer and Certified Scrum Coach who works for Rally Software. In this conversation Alan shares his insight on the state of Agile and Scrum. He also takes a stab at predicting what is to come in the future. This interview also includes some advice both for budding agile consultants and also for organizations that are trying to transform to an agile way of working.
Manoj: Agile has been here for a while; it is everywhere now – small and large organizations, distributed organizations, etc. What’s new in agile? Where is it going?
Alan: One thing I am glad of is that, in a certain sense, there is not a lot that is new. I think we understand it [agile]; I think it works and I really don’t want people messing with it a lot. Ken Schwaber is fond of saying that the whole thing that got us into trouble in the first place was our search for the perfect process. One reason that Scrum is so strong is that it is a process framework, not a process. It gives you all the important stuff and plenty of room to adapt.
It is true that agile does seem to be spreading everywhere. I saw a question in a trainer’s discussion group where someone was asking, "When is it going to be when the only option to do your project is agile? Then you don’t have to say you are doing agile or waterfall, you will just say we are doing a project." I think we will get to a state where that’s [agile] just the way to do things. Students will come out of school knowing agile, and they will know iterative and incremental makes sense; it will be just as logical to them as waterfall was to us when we came out of school. When I came out of school, I thought, what possible other way there could be to do a project? It [waterfall] was so logical. You start by collecting things [requirements]; you go thru this orderly process; what could possibly go wrong? We discovered what it was – what could go wrong.
Manoj: Many say agile has crossed the chasm. A look at the job board suggests that there are a lot of companies going the agile way. But many of them are more or less doing the same things they have been doing for years. They are not really getting to the state of hyper-productivity that Dr. Sutherland talks about. What is your advice to these organizations? In addition, what is your advice to a practitioner in that situation?
Alan: About the job boards: when I see some of those job descriptions, I just hope the people who are new to agile can read that description and notice the fallacy. Many of them will say they want agile people and then proceed to list all the anti-agile practices and philosophies there are. I also have trouble with the term “agile project manager.” To me – not to every one- to me, it is a little oxymoronic. I think agile works better with the mindset of release management than project management. I really worry for people going to some place just because they say they are agile and later finding out they really are not.
On the other hand, it seems like some people in the scrum community take offense at the idea that perhaps we don’t all have to win the gold medal and be hyper-productive. Perhaps we just want to be good or a little better than we were – that is enough. So I don’t take offense if somebody gets 50% out of scrum, where 100% is that hyper productive (8X, 10X) improvement in productivity. Sure teams should strive for that but I don’t think that is necessary for everyone. I think it is fine if you don’t do that. If you want to, you can. Then you have to be willing to make the changes that are required to get those kinds of results. You don’t get it for free.
I wrote a blog recently on Rally’s Agileblog named “Your Company Is Insane.” The theme of it is the old joke about insanity being defined as doing the same thing over and over and expecting different results. These companies will say, oh, go to this class to learn a new way of doing software. And then when you come back from the class and say we need to change, the company's response is, well we can’t do that. To me, that is insane - you want different results, but you are not willing to make changes. As a job seeker, you have to find a company that is willing to make those changes.
For organizations, it will make a lot of sense for there to be some experienced coaching assistance. Of course I am a coach so you will have to take this with a grain of salt. You might hire that person – it doesn’t have to be an external consultant. It really helps to have someone who has gone through this type of change and can guide you through it. You have to be careful about who you hire. Every ScrumMaster with a year of experience who got laid off last year is now on their own as an agile coach. You will get quite widely varying results depending on who you hire.
Manoj: What is your advice to an organization that is evaluating tools?
Alan: First of all, don’t buy a tool until you understand the process. Some companies might think that by using an agile tool they will automatically become agile. That really gets in the way having them understand the real process. Understand what you are doing first; learn the process. If you look at the literature of process automation, rule one is “before you automate the process, make sure it is correct.” There is nothing worse than automating a broken work flow at a great cost and then realizing that you automated something that is brain dead. So yes, make sure you understand the process first.
The second piece is making sure you know why you are looking for a tool. This is not different from any other kind of software infrastructure purchase. When I say make sure you know why, I am trying to be a step above the “create a list all your requirements.” We should start with asking why we are not just using post-its on the wall. There are a lots of good reasons why – part of my team is in Bangalore another part is in Colorado, and yet another in California – we are distributed; that’s a good reason. We use external partners, we both use scrum, we want to be able to use the same tool and track one another. That also is a good reason. So there are good reasons to use a tool. But reasons like “Post-its are messy” aren’t compelling.
There is a huge selection of tools now. It wouldn’t be good to get used to a tool and completely weave the tool into the process of my company, then have the company that created the tool go out of business. So as always, there is the trust and longevity issue when you are picking a tool. I don’t spend my time surveying the agile tool world. From what I know there are not many choices if you are looking for an enterprise-wide, distributed, roll-up kind of complete tool. Most of the tools out there are doing what you do can with the post-its on the board--for a team at a time, or for multiple teams. There are a vanishingly few with which you will be able to have integrated, roll up information all across your enterprise, that you can use to help do complete portfolio management. That’s a tall order. ... That [the complete enterprise-level capability] to me is the best reason – the most compelling reason – to look for an agile tool. You cannot do that on post its. You need to move up a level. The more you get to this larger enterprise level requirements; the fewer tools there are that support that. ...
Manoj: Organizational change is what you try to do as a coach. Remember the time when the organizations were going from functional to matrixed and then back again? How do you relate the agile movement to that?
Alan: I went through some of that. It was like a pendulum swinging back and forth; whatever you aren’t doing now will look better than what you are doing now. So you would just change from one to the other in some kind of cycle. I think what is different about agile is that none of those previous organizational changes came with a fundamental change to development methodology. You still did the waterfall, because that’s all we had. The fundamental parts of agile weren’t there. We were making all these changes but all the problems of the waterfall still existed. We were still required to have “accurate estimates,” which is of course oxymoronic. You were still required to have all the answers upfront, at the beginning of the project. You were still required to have plans far into the future even though it was not really possible. You still had a separate QA group who were totally oversubscribed, who never got the documentation they needed. That was very much like [rearranging] deck chairs on the Titanic – nothing fundamental was changing, just the cosmetic, just the atmospheric changes. That’s why I really think this [agile] is different. But only if the company dares to make the change. If you go agile but you don’t change your management structure--you keep your PMO, follow phase gates, all these other things--then you are limiting what you can do. You are not making the fundamental changes. Ken Schwaber once said, only 25% of the companies are realizing the potential of Scrum because most people are afraid to make those changes.
Manoj: Can you tell me about your career in this field; how did you get where you are today?
Alan: I started in software development a long time ago, before agile was even thought about in any way. So the first 25 years of my career were in waterfall environments at various levels, as a programmer, manager, director, or VP at various places. I was lucky enough to be at a company, Amazon.com, that in a certain way, without doing it on purpose, really embraced a lot of agile concepts. One of the things they do is that they very much allow developers and teams to own the “how;” own their own ability to build products. They [Amazon] are very careful about what they build; that often comes from outside the team. They hire really good people and say, figure out how to fix your own problems. My team at that time was not happy with the development process we had. So I went out and did some research and put together some options for the team. One of the options I gave them was Scrum. The minute I saw it I felt, “Oh, my God, this is it!” I couldn’t believe it. I probably sold it to them in a totally biased way – because I was the manager. But it worked for us. We did a horrible job of it at first but it still worked really well for us.
That’s how I got into it. I stayed in that product for three years. By that time, Scrum has really spread. It tends to spread if you allow it to. The developers smell it. One team gets it in your company. Other teams see them having a lot of fun and lot of success, in a way different from whatever they are doing. It spread a lot around Amazon. I was able to convince them [Amazon] to move me into a corporate resource - trainer and coach. So I did that for a couple of years. But there were some limitations and that made me want to move out of Amazon into the wilder and wider world of Scrum/agile consulting. So that’s what got me here.
Manoj: What advice do you have for someone starting out in agile/Scrum and looking at career path similar to yours?
Alan: The scary part about this industry is that it is all changing so much that it is really not clear what a career path will look like in the future--[one that is] predictable and dependable. I have been growing an opinion lately that the really non-trivial stuff is actually done from within the company. Consultants come in and they help, but somewhere in that company, someone engineers the actual transformation. And some of the people who contributed the most, the ones that really built something, do so as full-time employees of a company because it takes a long time to learn and then transform even to the point where it is not sticky yet. It is usually a multi-year undertaking. So the real non-trivial stuff happens within the company. It is not the consultant blogging, arguing, and splitting hairs about vocabulary and all that. So in a sense my advice is to get a real job! Learn how to do software development. Be a coder, be a tester, be a development manager, somewhere in the line that builds products. Make agile your tool and build around that.
I love going in as a consultant, and love helping companies transform, love helping teams start off, love teaching CSM. I love doing all the stuff but it is mostly in small pieces in different places. And the idea of transforming a company from waterfall to scrum, that is big. I see that coming from within not from outside of the company.
Manoj: If you are to predict where the agile movement and the software development process as whole will be in next 5 to 10 years, what would be your prediction?
Alan: I am not smart enough to answer that question! One of the reasons why XP, Scrum, etc., came about is the technological advances - Smalltalk and the like. Who knows where that will lead us? I don’t. I do think that things will get much less fragmented in the agile world. We will stop agonizing over all these variances that are out there. We will find combining them is not revolutionary. It will be what you do. XP leaves some wanting and Scrum leaves some wanting as well. But together they cover it.
And then, we need somehow to make the big organizational changes. There are so many organizational aspects that grew up within the waterfall environment that cause some friction with Agile. Aspects such as different management structures, different ratios of managers to individual contributors, the issues around yearly budgeting vs. agile budgeting, around performance reviews, compensations etc, etc. Some of those I hope start to smooth out in the next 5 to 10 years.