The Software Development Reality: The Need for Scalable Architecture
The Software Development Reality: The Need for Scalable Architecture
Why software development is slower than we expect?
At TechWavve headquarters, a new announcement has brought excitement—a green light for the upcoming merger. New exciting opportunities, substantial growth, a doubled customer base, and fresh market openings are on the horizon. This is going to be a game-changer.
During this executive team's excitement, a big cloud of concern over the head of engineering. Immersed in the ongoing complex projects, the engineering team cannot even comprehend the scale of the challenge ahead. Merging or transitioning the already complex software landscape with the software from another company is a formidable task, like scaling Everest.
What hinders the software landscape from supporting the desired business agility?
The complex reality of software development
In today's era of software development, numerous tools, technologies, and programming languages are available, simplifying development and making it accessible to a wider audience. With this modern technology, there is a prevailing expectation that the entire product development process should be more straightforward and efficient.
However, the reality for many companies doesn't quite match this expectation of effortlessly building software applications. Why is that?
When we look back at product development from a decade ago and compare it to the present, we must acknowledge significant improvements. Systems can now be built more swiftly, with better product quality and an incomparable user experience.
But despite these advancements, fundamental forces persist, making software development more complex than we expect.
Software development is more about maintenance than development
A seemingly minor addition to meet a customer's request involved just 10 new lines of code. However, this new code is expected to function reliably for the next 5, 10, or even more years, as long as it's in use by any customer in a production environment. Consequently, sustained maintenance of this code is necessary during these years, requiring a team member who thoroughly understands its functionality.
Software maintenance consists of post-release activities, including updates after the code has been released to production. This can involve addressing outdated features, improving performance, fixing bugs, or adapting to evolving technologies.
Unfortunately, the significance of code maintenance is often overlooked. Despite the belief that code is a one-time creation, the reality is that, at some point, this code will either malfunction or, at the very least, increase the cognitive load for the team.
Statistical data shows that software maintenance accounts for 80%-90% of the total software development effort. On average, on a yearly basis, software maintenance costs may range from 15% to 25% of the initial development cost. In simpler terms, to build an application and keep it live in production for at least 5 years, you should anticipate dedicating five times more effort (and cost) than initially planned for development.
Size and complexity matter
Each line of code introduced into a system necessitates ongoing maintenance. Consequently, the size and complexity of the software significantly influence the productivity of development.
That’s why creating a new system from scratch is typically a swift process. During the initial development phase, the focus is solely on construction, free from the weight of past code created by someone else. Developers are also enthusiastic and motivated to introduce a brand-new innovative feature.
However, the scenario shifts as the system expands, and the once-new code ages into what is commonly known as legacy code. The complexity of the legacy code is further compounded by frequent changes, making it less organised and understandable when new developers join the team.
Despite the team's eagerness to progress with fresh new features, the challenges of correcting issues from past code pull them back into the realm of addressing and maintaining the existing system.
Research has explored the relationship between system size and the effort needed for maintenance. The simplified formula within the COCOMO model is:
Effort = a * (Size)^b
where, "a" represents the productivity of the development team, and "b" indicates the scalability of the product.
Without delving into complex mathematics, it's clear that size significantly impacts effort. And even more, the more complex the software architecture, the steeper the growth trajectory in terms of required effort.
Software development beyond software programming
Software development goes beyond mere programming. Typically, the software implementation constitutes less than 50% of the total time invested in bringing the code to production.
The entire software delivery process involves more than just coding—it includes design, code testing, deployments, and handling operational issues. Maturing this processes and automation can significantly reduce the effort required for these supporting activities. However, achieving these improvements requires allocating time for technical enhancements, which is often challenging alongside the demand for new features.
Adding to this complexity, bringing a feature to production involves addressing new compliance and security requirements, aligning with stakeholders, and accommodating inevitable changes in requirements. Over time, the cumulative impact of these factors intensifies, especially as the system expands.
Business agility through scalable software architecture
In today's dynamic business landscape marked by swift shifts, mergers, and heightened competition, software systems demand agility. The speed of software development hinges on key factors – size and scalability. While optimising size through lean features is important, the natural growth of a system over time is inevitable.
Crucially, the scalability and complexity of a software system, rooted in its architecture, significantly impact agility. A large, non-modular system with complex code obstructs new feature additions and timely bug fixes. A monolithic structure managed by multiple teams cannot support releasing new features fast. The downsides of such architecture become clear as data volumes and incoming requests increase, leading to performance issues and rising scaling costs.
This blog is part of the series on Scalable Software Architecture. Below, you'll find an index of the articles in this series: