Incremental Design – Part 1

Continuous Delivery is all about making small changes. Work flows more easily, planning is simpler, error detection is helped and the time from idea to value is reduced when we make changes in small increments, but how do you solve big problems in small pieces? How do you maintain a coherent design when each change is small?

There are many facets to this problem. The organisations that are best at it tend to take a very broad view of design and think about things like how teams are structured, how work is defined and how design works when delivering a flow of many small features.
The traditional approach seems, on the face of it, sensible but it’s not. “Let’s think very hard and design everything in detail before we start”. Big-up-front design has a long history of not working very well, principally because at the point when you do the design you know the least about the problem that you ever will.
For a while we tried something else…

“We’re Agile We Don’t Need Your Stinking Design”¹

In the early days of agile adoption there were some very naive things said about software design. This was largely as a reaction to the failures of big-up-front design. Some of the advice used to remind me of the “President of the Universe” from Douglas Adams’ “The Restaurant at the End of the Universe”. In this excellent book, the advanced civilisations of the universe had decided, that the desire to become a politician should exclude anyone from becoming one. They had taken this to it’s logical conclusion and decided that the senior person should have no preconceived ideas about anything. So, the President of the Universe lived in a shack on a beach and started every day making no assumptions at all. On waking he would wonder at the big hot thing in the sky, muse on the nature of existence and so on. This left no time at all for any decision making, ideal for a politician perhaps, but not quite so good for a software developer.
Some people advised a similar approach to design. We should dump our assumptions and previous experience, and instead go straight to code and not think about design at all. If you weren’t sure what to do next, you should write a test!
Now there are few bigger fans of TDD than me, but I have always thought that this fear of design was stupid.
My own view is that everything that we, as software developers, do, is design. There is no distinction between coding and design. Coding *is* design, but it is not all that there is to it. Software development is an intriguingly difficult exercise in design and we should use all of the tools and experience at our disposal to accomplish it. Good design is our job, we should value it and strive for elegance and simplicity in all that we do.

WHITE PAPER

11 Black Holes of DevOps: How Not to Get Lost

Check out this white paper to learn the 11 DevOps “black holes” you can easily get sucked into… and how to avoid them!

A More Experimental Approach

If you are building a large complex system, or indeed any system, design is important, but we have learned through painful experience that trying to do all of the design up-front doesn’t work. Does this mean that we have reached an impasse?
Not really! Agile development practices are all about being experimental. So we should be experimental in our approach to design. If we want to be experimental what will that take?
Perhaps the most important thing is to allow for failure. By definition not all experiments succeed.
This need to allow for failure has several implications:

• If a design choice fails, how do we limit the impact of this failure?
• If we expect some of our design choices not to work, how do we tell?
• How do we organize our work to allow us to build upon what we have learned from our experiments?

Limiting the impact of failure, identifying what works and evolving our designs incrementally allows us to address that common concern, that early choices in design cannot be undone.
In subsequent blog posts I am going to suggest approaches to dealing with each of these. As a teaser here are some of the factors that I think are important in enabling a more incremental approach to design:
Team structure, Business-Alignment, Modelling and Models, Bounded Context, Isolation, Reactive Architectures, Loose-Coupling and Separation of concerns – there may be more.¹ A quote from Andrew Phillips – with tongue firmly in cheek.