How to remove waste in software development
In my previous blog post, I indicated that minimising waste in the software development process is one of the key components for successful product growth. In this post, I explain this in more detail: why is removing waste important in every process, what are the common types of waste in software development, and how to structurally approach the problem of waste removal.
Wasteful process can lead to enormous losses
Waste is any expense or effort expended that does not add value to the customer.
Waste is unwanted and unusable, and when not managed well, can lead to enormous losses. In the fashion industry for example, it is estimated that better waste management could save €4 billion per year up to 2030. Around 35% of waste occurs only during the supply chain before the garment reaches the end consumer.
I am not aware of such estimates in the domain of software product development. As this domain is a lot more abstract compared to manufacturing, quantifying waste is a far more difficult topic. I believe however that in software development, it is even more likely to have waste, because it is often hidden and not obvious to identify. Organisations who are aware of the importance of eliminating wasteful practices certainly benefit from higher productivity and more satisfied customers.
Common types of waste in software development
The term waste is closely related to the concept of lean manufacturing, a methodology that focuses on minimising waste and maximising productivity within manufacturing systems. The origins of lean manufacturing are based on the Toyota Production system, developed in the 1950s by Taichi Ohno, a Japanese industrial engineer. Taichi devised the seven wastes in a manufacturing system: Inventory, Overproduction, Extra Processing, Transportation, Waiting, Motion and Defects.
Lean as a philosophy has emerged over the recent decades and became an inspiration for all branches of industries. The idea of removing waste became so popular in the 90s that inspired lean software development as well as Kanban. The seven types of waste in lean manufacturing have been translated to seven types of waste in software development.
Although this translation captures quite well some common inefficiencies in software development, we should not fully rely on it, because software development is a very different domain from manufacturing. In manufacturing a physical product is built following a defined number of steps, while in software development we built something intangible following a very complex process. In manufacturing building a second product means repeating the same process, while in software development, the source code can simply be reused without the need to reimplement. There are types of wastes in software development that are not typical for manufacturing and vice versa.
In the table below I have listed the most common inefficiencies I have encountered in many organisations. The first seven of them originate from lean manufacturing, the others are common wastes but specific for software development.
How to remove waste in software development
Sometimes organisations integrate focused waste reduction approach. Employees dedicate a special period when they work on waste removal. This can be a useful practice because this is time when there are no other feature development priorities. However, as the time dedicated to reduce waste is usually limited, this period often results in cleaning the issues that are very easy to tackle, e.g. removing unused code, cleaning the backlog. Problems that require more crucial change, like redesigning a system, or adjusting the governance approach would remain unsolved.
Another practice is continuous waste reduction, when part of the product development (10–15%) is continuously allocated on improvements. The positive side of this approach is that it is a continuous process and it does not block the feature development. In practice however, it can happen that this 10–15% never get a priority due to pressing deadlines for delivering features. To make it happen, teams should have a good discipline and make sure to regularly include waste reduction issues in their sprint backlog.
While both approaches are valuable, what I often see missing is that they capture limited scope. In the first approach, the decision of which problem to tackle might be made by an individual, in the second approach, it is typically a team’s decision. During a retrospective meeting, the development team might identify a critical bottleneck in their work, but if this is not something the team could solve autonomously (e.g ineffective communication with another team or with the product managers), they would leave the problem unsolved.
And that’s unfortunate, because it might lead to spending effort on solving small issues while the basis is still wrong. The product development is a complex system and requires a joint effort between different stakeholders and teams. There should be an efficient flow of information in the complete system to make the work of this system optimal. One way to address this challenge is to form a team of diverse roles, who will make decisions about waste reduction in a more strategic and structured way.
Identify the wastes in your process
A good way to start with waste reduction is to evaluate each phase of the feature development process we follow. We evaluate whether each phase is performed effectively and efficiently and whether it produces high quality as output. The list of common wastes (Figure 1) can be used as a guidance, but we should not limit our thinking to only those types of waste. Each process is unique and might very well contain its own unique hidden wastes.
For example, the product development might follow the process shown in the Figure 2.
When we are evaluating the Development phase, we might for example discover a high percentage of user stories that are in “In Progress” state. This is one of the common wastes listed above — Partially done work.
The team should discuss and identify the reason behind this problem. It can be due to frequently shifting business priorities, lack of domain knowledge in the team, or simply inefficient backlog management. Although this issue might be discovered by the software engineers, the waste is caused in the previous phase, during feature requests prioritisation, and should be addressed by the product owners/managers.
Another example might be a high number of defects discovered in the Acceptance testing phase. This is a sign that there is insufficient testing in the previous stages. By observing the types of defects identified in Acceptance testing, we could discover that there is insufficient unit testing or integration testing. Once the waste is identified, the solution becomes obvious — e.g. adjusting the test strategy and increasing the code coverage threshold for unit tests.
Next step is prioritisation. This is an important step and should not be avoided. The whole process of waste reduction causes effort, so this effort should be spent in the most optimal way to get the best outcome.
We could use a simple prioritisation technique, as shown in Figure 3, where each waste identified is assigned Impact value and Effort value (the effort required to eliminate the waste). Once we have the wastes plotted in this matrix, we prioritise, by choosing those that are with high value and reasonably low effort. We could for example select the top three types of waste to be solved and integrate them in the backlog.
Of course, making an accurate quantified estimation is usually unfeasible, but it is also impractical and unnecessary. It’s sufficient to prioritise wastes based on our best guesses.
Removing waste is a continuous and gradual process. People have the natural tendency to accommodate easily to wasteful practices, so setting up a continuous process that integrates lean thinking is important.
Waste usually happens due to making mistakes or unawareness. But as long as we are learning from these mistakes, this is not really a waste. It becomes a waste if we stay ignorant to these mistakes and we don’t do anything to make an improvement.