Waterfall? Get Over it Already!

Is it fair to call Quasimodo ugly?

Earlier this year, on the LinkedIn Agile and Lean Software discussion group, someone posted the question, “Is it fair or accurate to malign the waterfall process as is rote when people push agile and Scrum?”  The original question drew a lot of discussion, and then it seemed to die down.  All of a sudden it has reared its ugly head again, and really requires a more thorough response.

When I first saw the post, I laughed.  Here was a guy posting in the Agile and Lean Software group a question which implied that any negative talk about waterfall was “maligning” it.  Needless to say, the wording implied an assumed answer.  My first thought was that this guy was a troll. And the usual folks would quickly send him on his way with scorn and derision.

What surprised me, and somewhat delighted me, was how many people replied with thoughtful discussion.  Folks took the question, wording and all, seriously.  They started to explore the question of whether agile had become dogmatic, or perhaps we were being unfair to waterfall.  The conversation was going to help everyone better understand why the world is moving to a better way of creating software, which is what we are calling agile (and now also Lean).  This is, after all, what discussion groups do so well.  We learned so much through the discussion groups in the early days of agile, as if we were our own Socratic society.

I quickly became disillusioned though.  The discussion was not a serious examination of the pitfalls of the processes known as waterfall management, and how to overcome them by moving to agile development.  Nor was it an open conversation on why – when it comes to software development projects -waterfall is the poorest of choices.  Instead, we saw a lot of folks very nicely saying that waterfall management will work just fine if you have all of the information up front… or if you have a very clearly defined set of goals and a strong understanding of the problem domain.  This would lead one to say, “gee, if that’s all it takes, why learn all these new processes?  Maybe we should just put a lot of time and energy into being able to get all the information up front, or into making sure we have a clearly defined set of goals and ensure they just don’t change during the project?”  Well I’ll tell you why.

***It just doesn’t work!***

This is a good time to remember why agile software practices came into being in the first place.  We didn’t all sit down and say, “You know, everything is going so well and projects are really coming in on time/budget, with high-quality code.  Let’s change everything.”

This all started because folks realized that projects were failing more often than succeeding.  Most of the projects that were considered success on the outside were the result of long hours and dangerous shortcuts.  Change became something to resist, even if the change was the right thing to do.

We spent a lot of time and energy trying to find ways to change this; we usually exhausted time and energy that would have been better spent creating software.  Agile development is about recognizing the difficulties and complexities of the software world, accepting them and working in a way that harnesses the ability to change software at a minimal cost.

So no, everyone doesn’t get a trophy.  Waterfall is not a good way to approach software projects.  That’s ok.  There are many non-software projects that would do quite well in the waterfall model.

Meanwhile, let’s dedicate ourselves to spending this coming year getting the most out of agile, especially finding the time to improve the craft of creating the software. Arguing about whether waterfall is still good is *so* 2011!