The Agile Coach on Production Support

One of the biggest struggles I’ve seen in organizations adopting Agile is in the area of Production Support. Every organization that has a product to support has to manage this. It’s a critical part of the business. There’s a lot of responsibility to manage a 24/7 Production environment. The middle of the night calls, working on code you didn’t create. Speediness, precision, reaction time and problem-solving are crucial.

The struggle typically comes in the form of pulling people away from active Scrum teams to make whatever hot fixes are necessary. This is clearly disruptive to the Scrum development team. Oftentimes, a team member is being pulled away to work on something that’s very unpredictable. Who knows how long they could be gone from the team; an hour, a day, a week? This disrupts our rhythm and velocity. It makes planning difficult. Team cohesiveness suffers. The person getting yanked to do the Production work usually isn’t too happy either. Context switching makes them less productive. And in the end, we actually get less done!

Alas, we need to get these ongoing high priority bug fixes out the door asap. And we need the ability to re-prioritize any time. We need focus. And we know that 2 week sprints aren’t a good fit.

So how do we combat this? How can we keep our Production environment up to speed, while also allowing our Scrum teams to develop that stuff they’ve committed to getting done for their Product Owner?

My preference, and one that I’ve seen work well in many organizations, is to create a new Production Support team that works these hot fixes as part of a Kanban effort. The only negative feedback I’ve heard on this is that it seems like punishment to some (think of the Gulag in Northern Syberia). To combat this feeling, we’ve tried cycling folks on and off the team so they don’t burn out. As any good Managers knows, it goes a long way to show your appreciation to this group in some way. Maybe a really nice work environment, new laptops, snacks/drinks, etc. A sincere pat on the back also works wonders. Get creative. Additionally, there may be some folks that don’t fit well in your new highly collaborative team culture. They’re very smart folks, but more of the lone wolf variety. This team could be a natural fit for them.

At the end of the day, technical debt is something that will keep us in this death spiral. We want fewer bugs in Production. In order to get out of it, we need to improve our technical practices, our code quality, our testing, our architecture, and our design. It’s hard to make any progress if we keep patching up something that’s not good to begin with. Sometimes it’s just easier in the long run to scrap it and start fresh (I know, that’s a hard pill to swallow).

How do you handle Production Support in your Agile environment?