Unlocking the architectural scalability of software products
Unlocking the architectural scalability of software products
In my previous post I discussed about the key components essential to enable sustainable product growth: product strategy, product team governance, product development processes and technology. This blog post focusses on the last component — technology, and how to achieve a scalable technical architecture that enables product growth.
A scalable technical architecture grows together with the business
In the technology world, architecture refers to the technical decisions made when building a product that are difficult to reverse. Selecting a technology or defining the high-level building blocks of the software product are examples of such decisions. It is like architecting a house and deciding on the wall materials or the main layout of the house.
Scalable software architecture is architecture that does not only fit our current needs, but has the ability to support our business in the long run.
To understand the importance of a scalable architecture, let’s take an example. Imagine we’ve built a new product and we have only a small customer base at the moment. The product gains traction, and we get increased customer demand. This demand comes with a responsibility: we need to extend the product quickly with new features or enhance the existing features. To support these changes, having a scalable technical architecture is crucial. Without scalability in place, every change made in the system would be highly costly and we could easily end up in a situation where fully rebuilding the product is the only way forward.
At some point, our customer base grows 10x. The number of users accessing the system simultaneously increases. Without a scalable architecture, the performance of our product would degrade under the increased load. The end user would face slow system responsiveness or a system failure. Only adding more CPU power is rarely a good solution. A cloud migration approach will help with infrastructure scaling, but if the software is not designed with scalability in mind, the benefits of cloud would not be enjoyed, and the costs of licensing would grow unnecessarily high. Redesigning the system (or even rebuilding it) would be a way forward, but that is a burdensome and costly solution.
What are some symptoms of having non-scalable architecture
While the negative effect of having a non-scalable architecture will mostly show in the long run, there are some symptoms to observe showing that we are running the risk of having non-scalable architecture.
There are no (or vaguely defined) non-functional requirements of the product we are building.
We don’t have insight in how much the non-functional requirements are satisfied during the product implementation.
We don’t have insight in how much the product is used in production.
We have little insight in the current implemented architecture.
Adding new product features are much more complex than expected.
Performance degradation is reported when the system is under increased load.
Building a scalable architecture in an agile world
It is expected that architecture decisions are correctly made early in the product development, as reversing them can be significantly costly. However, this can be contradictory with today’s world where everything is agile and there is a high unpredictability of the future.
According to Gartner (From Fragile to Agile Software Architecture, 2019), modern agile software development methodologies provide little guidance on architecting. As a result, teams are left applying traditional architectural practices on agile projects, which does not work well in practice.
What we need to understand is that great architectures do not happen over night or in one go. Architecting is a gradual process that requires iteration, feedback and adjustment. Organisations like Uber or Spotify are known for their flexible microservices architecture model. However these architectures have not been designed upfront.
We think of evolving a microservice architecture more like “trimming a hedge” so that it eventually grows correctly, rather than a top-down or one-time architecture (or re-architecture) effort. It’s a dynamic and progressive process.”— Adam Gluck, Uber Engineering.
Best practices to build an agile, yet scalable architecture
Here are some basic practices on how to build a scalable architecture in an agile world.
1. Define architecture principles with the business goals in mind. Architecture starts with a good understanding of the business goals. Who are the end customers and what expectations do they have? How many users would use the system? Do we expect this number to scale in the future? Are there peak periods when we should expect an excessive number of requests accessing the system? What is the business impact of a system failure? Is tracking historic customer data necessary? Does the system handle confidential data?
The business goals drive the definition of architecture principles, key quality rules and guidelines that the implementation should follow. For example, Availability, Ease-of-Use, Data Security, Performance Efficiency might be a few of them. These aspects should not only be defined on paper, but they should be embedded in the product organisation. Designing a system that needs to be fully operational 99.999% of the time is different than a system where an hour down time would not have a significant impact to anyone.
2. Define only a high-level initial architecture model. Next step is developing an initial high-level architecture model. We start from defining the key business capabilities and a technical architecture layout that reflects these capabilities. We define the key high-level components, and how the data flows between components. The architecture layout should be designed with the architecture principles in mind. If high Availability is an architecture principle, adding a component that all other components rely on might not be a good decision.
Defining the initial architecture model is a technical visionary exercise led by the architects. They set up the vision with the assumption that it will be followed during the implementation. Going too much in detail in the beginning stage can be dangerous. The initial architecture model should be kept simple. Overthinking leads to unnecessarily spent time, and even more, driving the implementation in the direction that might later show to be a wrong decision.
3. Build the software in a modular way. Modularity is one of the principles for architecture design that I would like to highlight, as this is a general principle that almost always applies. Modular architecture makes maintenance of the product easier and lowers the risk of introducing defects. It is therefore directly related to the maintenance spend.
Modular architecture means that the software is built in small, loosely coupled components. Modularity as a principle should be followed from top level (high level components) down to the smallest units (methods or procedures). Modular architecture means that the system is nicely structured in smaller components with minimal dependencies between these components and where each component is further nicely structured internally. With a modular architecture, every module could be reused or replaced while causing a minimal impact to the rest of the system.
Note that modular architecture does not necessarily mean service-based architecture. Services can add great value but when introduced in the right way and at the right time. When introduced too early (for example, when the product and the development team is still small), a granular service-oriented architecture could introduce unnecessary complexity. A modular architecture is however still recommended. When a single monolith is built in a modular way, its maintenance is easier, and at some point in time, its modules could easily be separated as (micro)services.
5. Embed non-functional testing Non-functional requirements are system requirements that are not related to any functionality of the system. Instead they refer to the system quality, and should be associated to the architecture principles. While the architecture principles are high level guidelines, non-functional requirements are concrete and ideally — measurable requirements. For example if high performance is a principle, a non-functional requirement might be defined as “the maximal response time of the every page is 2 seconds” or “the external service should respond in 1 second”.
To ensure that the implementation meets the non-functional requirements, non-functional testing should be integrated in the development life cycle. That is why the non-functional requirements should be as concrete and measurable as possible, so that an anomaly of the implementation can be discovered during testing, before the system reaches the end user. If we test the page responsiveness requirement for example, test scenarios should be implemented that simulate the system behaviour under realistic load. The non-functional testing should be done as mush as possible in an automated way.
6. Observe the evolution of the architecture. Тhe architecture is the technical vision of the product set by the architects, which is supposed to be realised by the software development teams. In practice however, the envisioned architecture is rarely consistent with the actual implemented architecture.
To minimise this gap, it is recommended to regularly observe the evolution of the architecture. Best is to use tools that provide architecture visualisation with the source code as input (e.g. Code Maps, Sigrid, Structure 101). The benefit is twofold: the architecture visualisation is generated, with no need to input data manually; and importantly, the visualisation represents the “actual” implementation. Any other visualisation is just a mental model of what we expect the architecture to be like.
These tools however, come with limitations. Today’s systems are built using different technologies. A fully automated architecture visualisation tool is rarely able to capture correctly the architecture if the system is built using diverse set of technologies where components built in different technologies communicate with each other.
From organisational point of view, it is crucial to set a governance where there is a close alignment between the architects and the development teams. The developers are those who deal with the detailed practical problems. They need to have a good understanding on the architectural vision, as well as give their feedback on what works or not in practice.
7. Revisit the architecture periodically. The business goals may change over time, the team size and structure may change as well. It is therefore necessary to revise periodically the architecture and the guidelines we have set, and to make an adjustment if necessary. For example if over time we observe that a component continuously grows in size, it is a good idea to rethink the design. We might discover that part of this functionality could be nicely isolated in a separate component. This way of thinking could be implemented on a system architecture level (where discussions could be held with every sprint retrospective), but also on a higher enterprise architecture level (where such reviews could be done once in a few months).
It could happen that we start with one envisioned architecture in mind and we transition to a completely different one. This transition however, should be gradual and should naturally evolve. Our role is to keep observing this evolution and adjust where necessary, while setting appropriate rules or principles that tame the complexity and keep it manageable.
This blog is last the series on how to achieve sustainable digital product growth. Here’s an index of all the articles in this series: