Continuous Improvement in Software Development

I already wrote about „The Killer Difference between Mechanical Engineering and Software Engineering“ and „Similarities in Quality Assurance between Cambodian Silk Painting and Software Development“. Related to this some comments on continuous improvement in software development and refactoring.

As mentioned earlier, there is not classical production process in software development. In classic production, we have repetitive work and quality measures like Cp and Cpk which are monitored over time and if they are out of certain limits (warn limits and out of specification limits), actions are taken. Over time this leads to a better production process by continuous monitoring and continuous improvement by taking the metrics from the monitoring process. In software development we need to do something else. We need a new approach (see post about the difference on mechanical engineering and software engineering).

During software development we need to use raw models to guide the development like source conventions, design patterns, and standard designs for error handling and such stuff. A clean, layered architecture also helps to guide development and leads to good code on green field projects (see the article about QA for silk painting).  But, as soon as the already present code is changed, the code degrades slowly, but steadily. Here a totally new approach needs to come in: We need to improve the existing code as steadily as the code degrades. As the entropy in physics tells us: The entropy is increasing steadily as long as there is no energy used to decrease it. That is why we need to put a lot of effort into the permanent improvement of existing code.

The buzz word for this process is „Refactoring“. It is needed to have a fixed amount of time planned for continuous improvement of existing code by refactoring. I suggest an amount of 5% to 10% of time as a minimum for each project. Depending on the actual state of the project it may be needed to plan more. Up to 50% is realistic if a new brown field project is  started (a new development is started on existing code) to smooth the way for the new project. There are some points to consider:

  1. Is the refactoring done independently? Are some people planned to do only refactoring on the code independently of the other developers? This can be done to change the architecture or some designs. Exchanging frameworks, cleaning up parts of the software or separating some parts from another are tasks which can be considered to be done, too.
  2. Is the refactoring done during development? The refactoring can be done during the development of new functionality. This approach needs a strong mindset of the developers. For each new functionality to be implemented, the code is evaluated first and refactored to provide a clean ground for the new functionality. It is better to clean the effected parts first, before adding new functionality.

Both approaches have their advantages and disadvantages. I suggest to do both: It is best to train developers to do some refactoring before new functionality is implemented to get a clean implementation of new functionality. These refactorings are allowed to be made after it was clarified that enough tests back up the changes. If there are not enough: These tests need to be created. In the daily standup each developer needs can tell her colleagues which parts are to be refactored to avoid conflicts due to simultaneous code changes. Additionally, a senior developer or architect may be needed to be asked for approval if the changes are more risky or effect too much code.

Independent refactorings should be added as tasks to the backlog as soon as they touch the overall design or architecture. These refactorings should be planned separately, but it is to be assured, that they are not forgotten. A good backlog is needed for that and a good set of rules on how to determine the priorities for each ticket.