This post is from the Collabnet VersionOne blog and has not been updated since the original publish date.
Overcoming Information Overload in the Agile/Lean World
I recently attended Agile 2013 conference and tradeshow at Nashville, TN. It was a great! I always visit the “agile bookstore” at such conferences, and didn't miss it at Agile 2013. Most of these books in the physical bookstore would be subsumed and vastly outnumbered by the Amazon.com bookstore. After all, the physical bookstore probably had around 150 distinct titles compared to Amazon.com’s massive number of titles on agile and lean topics. To my surprise, I found that many titles in the Agile 2013 physical bookstore were priced competitively with Amazon.com, and there were few titles at the physical bookstore that are not available on Amazon.com! I spent an hour browsing through the physical bookstore and bought two books out of 150. There were several more I wanted to buy and read, but my reality check of “lack of time” stopped me cold. Where would I find the time to read, digest, and more importantly, actually apply the wisdom in so many books in my working life? Needless to say, this physical bookstore was a very tiny sliver of what is available on agile and lean topics. Consider the agile/lean “information fog” around you:
- Large number of published books: On Amazon.com there are approximately 2,500 titles with the word “agile” and approx. 630 titles with the word “Lean software” in it.
- Several thousand scholarly articles and conference proceedings
- Hundreds of blogs authors publishing thousands of blogs
- Several dozen newsgroups
- Large collections of on/line slideware and documents
- Several hundreds of agile/lean web sites
- Hundreds of thousands of tweets and social/media communication
- RSS feeds
- YouTube videos, and of course
- Communication with your colleagues, agile/lean experts and thought leaders.
This information flood and overload can be overwhelming, but is not unique to the agile/lean field. It is practically in almost every field of knowledge. The amount of information available and accessible has grown by several orders of magnitude in the last 15 years and will continue to grow. However, as reported in a Forbes magazine recent article, our ability to consume that information hasn’t changed much. There are many efforts underway to deal with the information overload problem ranging from university research, recommended practices, various utilities and tools (filters, information agents, search engines, web mining agents, etc.), and a range of applications and mobile apps. In fact, the information on how to manage information overload creates its own second order information flood! Try to Googling “information overload” and you are immediately overloaded with more than 7m hits in 0.05 seconds! Information overload is one of the biggest irritations in modern life and agile/lean information overload is no exception. It creates anxiety and stress among knowledge workers and reduces their productivity. One of the sites I find useful to deal with this second order information flood is the site of the Information Overload Research Group, which is dedicated to reducing information overload, a problem that diminishes the productivity and quality of life of knowledge workers. Before we become enamored with information overload management approaches, methods, tools and apps, I would like to attack the problem of agile/lean information overload management by asking few critical questions: What is your end goal in acquiring and applying broad general agile/lean information? This kind of information covers topics such as Agile Manifesto, Agile values and principles, Lean principles, Scrum principles, cross/functional and self/organized agile teams, value flow, work in process (WIP) and cycle time, continuous improvements, iterative and incremental development, agile estimation, agile planning, agile metrics, etc. Anyone involved in the agile/lean development lifecycle must know these basics irrespective of their primary role (see below). These basics are available in most text books on agile and lean frameworks and methods.
- Is gaining the general broad information an end goal in itself? Or is that a means to an end?
- What are your end goals in acquiring and applying the general broad information?
- How much time do you have available for acquiring and applying the general broad information?
- How much effort are you willing to put in order to acquire and apply the general broad information?
What is more important is to learn how to apply the basic information to your specific situation and issues to achieve your end goals. This knowledge specific to your question is not something you are likely to find “off/the/shelf” in some book or on a web site. Your best approach is to engage an expert, either in/house expert or an external consultant with appropriate qualifications and credentials to help you apply general information to your specific situation, constraints, organizational culture, etc. For example, the general body of information will tell you that success in agile development requires self/organized, cross/functional teams and may offer general suggestion to achieve that goal. But you are more likely to need the help of an expert consultant or in/house expert to determine how to overcome obstacles in your organization to the formation and proper functioning of self/organized, cross/functional teams and how to effect the change from top/down management/led projects to self/organized, cross/functional team efforts. What is your end goal in acquiring and applying deep specific agile/lean knowledge? Assuming that you have acquired broad general information, do you want to acquire and apply deep specific knowledge in a particular sub/field of the agile/lean world? Although, the general broad information and basics are good starting point and are a prerequisite, they may not be enough in themselves. You need to acquire and apply deep specific knowledge starting with an attempt to answer these questions:
- Is gaining some deep specific knowledge an end goal in itself? Or is that a means to an end?
- What are your end goals in acquiring the deep specific knowledge?
The deep knowledge you may want to acquire in a specific sub/field needs to be tied to your primary role in the agile/lean world. Table 1 shows several examples of deep/specific knowledge in various sub/fields (practices, methods, case studies, patterns) of agile/lean world and how those are tied to specific primary roles. For example, “Agile/Lean Adoption and Transformation” knowledge and responsibility of applying that knowledge is primarily the responsibilities of executives, managers (functional, program, project managers) and ScrumMasters. DevOps and Continuous Delivery knowledge is primarily the responsibility of software developers and operations/IT staff. This table is not exhaustive by any means but is meant to illustrate specific sub/fields (shown in columns) and primary roles (shown in rows). How do you apply agile/lean knowledge to achieve your end goals
- How do you envision applying either broad general information of the agile/lean world or deep knowledge in one or more sub/fields of the agile/lean world to meet your end goals?
- What specific issues and challenges do you anticipate or have already experienced that you need to overcome in applying the knowledge?
- What is your success criterion?
Admittedly some of these questions raised above may be difficult for you to answer immediately or in the near term. They are meant to get you thinking in the right direction. Often until you get introductory knowledge of any specific area and try applying that knowledge, you will not know whether that knowledge would help you in your specific situation. However, if you don’t begin to answer these questions as your starting point, you will be more likely to feel overwhelmed with ever expanding and growing information flood in the agile/lean world. In other words, the lenses of specific deep knowledge in one or more sub/fields of agile/lean world and your specific role will help you achieve a sharper focus in acquiring knowledge that you can apply to reach your goals. In effect, a sharper focus of inquiry will help you deal better with the information overload. I will address some of the specific questions listed above, and offer potential approaches, guidelines and examples for managing information overload in the agile/lean world in future parts of this blog series. Stay tuned. Meanwhile, I would like to get your feedback in responses to these questions:
- How do you perceive and experience information overload in the agile/lean world?
- Can you answer some of the questions I have posed above?
- Can you share with us your current solutions to information overload in the agile/lean world?
I would love to hear from you either here or by e/mail (Satish.Thatte@VersionOne.com) or hit me on twitter (@smthatte).