You can’t schedule a breakthrough.

Roadmaps are designed to align teams and drive execution. Traditionally, they answered one question: What will be delivered by when?

Today, many organizations use outcome-driven roadmaps — focused on impact rather than features. A meaningful step forward. But even outcome-driven roadmaps are still instruments of execution. They optimize toward defined objectives and assumed directions.

Innovation begins earlier. Before the outcome is defined and the metrics are known.

It begins with uncertainty and exploration. With the question: What if…?

This is where organizations get stuck. They want innovation — but their operating model is built primarily for execution. And execution is hungry. It consumes time, talent, and unprotected ideas. The system is optimized for today — not tomorrow.

Common Reasons Why Innovation Fails

Organizations optimized for execution unintentionally suppress exploration. Their systems reward predictability, efficiency, and delivery — not discovery.

Three patterns come up repeatedly:

Execution consumes all capacity. Roadmaps, targets, and operational pressure expand to fill every available hour. Exploration becomes something to do later, when things calm down. In reality, later never comes.

Innovation becomes an event. Hackathons and idea days create real energy. People get excited, ideas flow, and for a moment it feels like anything is possible. Then Monday arrives. Without a continuation plan, ideas fade and everyone returns to their real work. Until the next hackathon.

Innovation gets detached from reality. Innovation labs and dedicated task forces are a genuine step forward. But the real risk is drifting too far from real users and real constraints. They produce interesting concepts that struggle to land back in the core product. The ideas can be good. The distance is what makes them impractical.

In all three cases, innovation isn’t rejected. It just doesn’t have enough space — or structure — to survive.

The Innovation Track

An Innovation Track is a deliberately protected space alongside your delivery roadmap — designed for exploration, not execution.

It exists to investigate opportunities that don’t yet fit into quarterly targets, but could materially shape the future of the business. It may explore emerging technologies, new customer segments, new value propositions, or shifts in the business model itself.

Its purpose is simple: test what might matter next — before committing the organization to build it.

It is time-bound, hypothesis-driven, and importantly, focused on real problems. It’s a structured way to reduce uncertainty before an idea enters the roadmap.

What Makes It Work

A small, focused, startup-like container inside your company — built to explore one meaningful opportunity with intention. An effective Innovation Track has three key components: a clear charter, the right people, and a deliberate time window.

The Charter

A charter is a simple one-pager that sets direction without prescribing output. It should capture:

The Team

Innovation doesn’t require a large team. In fact, small team learns much faster. The ideal setup looks like a startup founding team: 1–3 people, cross-functional by default, empowered to decide without permission, and connected to real users.

Small, sharp, and autonomous. That’s the formula.

The Time Window

Innovation Tracks are intentionally short. Four to eight weeks is the typical window — long enough to learn something real, short enough to avoid drifting. The expected outcome isn’t a polished feature. It’s a decision: do we continue, pivot, or stop?

Protecting the Track Without Isolating It

An Innovation Track only works if the team can explore freely — without being pulled in day-to-day execution. But there’s an equally dangerous failure mode: complete isolation so that the work loses touch with reality.

The art is in building a protective bubble with a door.

Protected from delivery pressure. No sprint commitments. No last-minute requests. The team needs to be temporarily removed from the urgent dealines so they can think clearly and be creative.

Stay connected to the real world. Exploration without grounding becomes fantasy. The team must have access to engineering realities, domain expertise, business stakeholders or real users. This is what keeps the Track connected to the real world.

Join only the rituals that matter. Monthly demos to share learning. A regular check-in with the sponsor. Visible to the organization — but not interrupted by it.

Make expectations explicit. Everyone around the Track should know what the team is working on, how long it runs, and why they aren’t available for delivery work. Clarity protects the team better than isolation.

Governance: Keeping It Light

At the end of every cycle, gather the team, the sponsor, and a few key stakeholders. The question on the table is straightforward: what did we learn, and what does that learning mean?

The conversation centres on three things: the story of the work, the strength of the signal, and the path forward. The decision itself is:

• Continue — the signal is promising enough to dig deeper.

• Pivot — the opportunity still feels real, but the hypothesis needs to shift.

• Pause — the idea is sound, but timing or dependencies aren’t right.

• Stop — the exploration served its purpose. But the idea isn’t worth pursuing further.

From Exploration to the Roadmap

Graduation is the moment an idea stops being a possibility and starts becoming a plan. An Innovation Track should never drift into delivery by accident — graduation is a conscious, evidence-based step.

A Track is ready to graduate when three things are true:

• there is a real signal (not a perfect metric, but enough evidence to believe the opportunity is meaningful),

• the idea is buildable (the team understands feasibility well enough to move into structured experimentation), and

• there is a value case (you know who benefits, how, and why it matters).

From there, the work becomes more tangible. It’s not yet time for full delivery, but for more structure than exploration. The team defines the smallest meaningful version, builds early prototypes, validates core assumptions with real users, and identifies early risks.

Once that work shows consistent value, the idea is ready for the roadmap: ownership defined, scope agreed, investment committed. Good graduation is about commitment informed by evidence.

How to Start in 2 Weeks

Getting started doesn’t require a special team, a new budget cycle, or a grand announcement. You can launch your first Innovation Track in two weeks — with the people and knowledge you already have.

Step 1 — Pick the opportunity. One problem or emerging question that keeps resurfacing. The kind of thing people say “we should look into this” about, but never do. Frame it as a question worth exploring, not a solution to build.

Step 2 — Form the team. Identify 1–3 curious, cross-functional people. Not the busiest — the most motivated. Nominate a sponsor who will protect the team and own the graduation decision.

Step 3 — Write the charter. One page. The problem, your hypotheses, the signals you’re looking for, the boundaries, and the timebox. Do it together in a single session. That conversation is the first act of exploration.

Step 4 — Set the conditions. Give the team temporary protection from delivery work. Make expectations explicit to everyone around them — what they’re working on, how long it runs, and why they aren’t available for other requests.

Step 5 — Start exploring. Let the team run. Talk to customers, review data, test early assumptions. The goal isn’t a polished output — it’s to start reducing uncertainty.

The first governance conversation happens at the end of the cycle, 4–8 weeks in. That’s when you decide what comes next.

The Real Question

Innovation thrives in a space that is intentional, protected, and connected — where ideas can be tested without pressure, explored without drifting, and evaluated with evidence.

The organizations that sustain innovation through scale aren’t the ones with the best ideas. They’re the ones that built systems where good ideas can survive long enough to become something.

The roadmap will always be there. Execution will always feel urgent. The question for every product and tech leader is whether they’ve built anything to protect the space before the roadmap — the uncertain, generative space where the next version of the company is beginning to take shape.

Subscribe now

Enjoyed this read? Subscribe to Lean Product Growth for regular updates on building and scaling a successful product organization.

But far too often, it’s treated as a technical concern, not a strategic one.

In reality, technical debt is a business cost — one that compounds silently until it blocks product progress.

Software Is More About Maintenance Than New Development

👉 Data shows that 70–80% of a software product’s total lifecycle cost goes to maintenance.

Not new features.
Not exciting launches.
Just… keeping the current system alive.

That includes:

• Troubleshooting fragile code

• Rewriting messy logic

• Updating dependencies and adapting to changing APIs

For every line of code you ship, expect to spend five times more effort maintaining it over time than it took to build.

Size and Software Complexity Matter

As complexity builds, teams slow down. And because the slowdown is gradual, it often goes unnoticed — until it’s too late.

Every new feature means more code.
And more code means more effort to maintain.

This is why early development feels fast: there’s no legacy, no detours.
But as the system grows, so does the burden. Developers inherit old decisions. Logic becomes tangled. Even small updates start to feel risky and slow.

The more complex the architecture, the steeper the growth curve in effort required.
Teams want to build forward — but are constantly pulled back to fix the past.

👉 Studies show that maintaining poor-quality code can require 2–3× more effort than working with clean, well-structured systems.

So Why Do We Delay It?

Because the cost of not addressing tech debt feels invisible — until it suddenly isn’t.

Teams push it aside because:

• “We’ll fix it next sprint” (but we don’t)

• “New features have a priority” (but they take longer each time)

• “It’s not urgent” (until we’re blocked)

The longer we delay, the harder — and more expensive — it is to fix.

Making the Call: Refactor, Rebuild, or Leave It?

When tech debt builds up, you’re not facing a code issue anymore — you’re making a strategic call.

Here are the three most common paths forward:

• Do nothing, accept it and leave it as is.

• Refactor parts of the system

• Rebuild in parallel

Each path has trade-offs. What you choose depends on timing, team maturity, product lifecycle, and business goals.

👉 Want the full breakdown?

Read the deep dive here: Technical Debt Decision-Making: Refactor, Rebuild, or Stand Pat

Have you made a business case for refactoring or rebuilding?
What worked? What didn’t?

Reply or comment — I’d love to hear your examples.

Enjoyed this Article?

Subscribe to Lean Product Growth for regular updates on building and scaling a successful product organization. Don’t miss out on insights, strategies, and actionable tips—delivered straight to your inbox.

Subscribe now

In early-stage companies, scrappy workflows are a strength, simple slack messages, spreadsheets are sufficient. But once a company starts to scale, those same workflows become friction. Coordination slows. Quality drops. People duplicate effort or lose track of who’s doing what.

Now, with AI reshaping the landscape, many companies are turning their attention inward. They’re asking: How can we improve internal productivity without adding overhead?

The answer isn’t more tools. It’s applying product thinking to how your business runs.

When you treat internal operations like a product—with users, feedback loops, testing, and continuous improvement—you don’t only digitize. You simplify. You identify hidden gaps and unlock opportunities. And you build effective systems that scale.

This article shares how to bring product principles into your internal operations so your company can scale with clarity instead of chaos.

Recognize Your Superpower: Your Users are Right Here

One of the biggest advantages of building internal tools?
Your users are sitting right next to you—or at worst, one Slack message away.

You don’t need formal research studies or stakeholder discovery calls.
You don’t have to navigate complex client org charts to arrange a call with a user.
You have direct access to the people you're building for—every day.

It’s a product manager’s dream: a direct line to your audience, in their own environment, every single day.

Bring the user into the process. This makes the development much more effective, the adoption more organic.

• Co-create user journeys to uncover friction points.

• Share mockups and wireframes for validation.

• Run quick feedback sessions.

• Define clear success criteria (not just "it works").

• Track real adoption, not just release dates.

Apply the same rigor you'd use for customer-facing products. You'll earn deeper engagement and build trust across teams.

Start With Discovery: What’s Actually Happening in Your Operations?

Many companies digitize their operations by taking what already exists and building a tool around it. They might hire a consultant to collect process documentation and define product requirements out of it.

That’s a mistake.

Before you automate or rebuild anything, understand the real workflows. Not the ones written in documents, but the ones happening in chats or email threads.

Start with structured discovery:

• Shadow internal users: Observe how people actually do the work. Where do they get stuck? What do they skip?

• Map the journey: Visualize the end-to-end flow. Where are the delays, handoffs, or points of failure?

• Surface pain points: What slows people down? What do they avoid doing? Where are they relying on ad-hoc fixes?

When we started our discovery, we uncovered huge opportunities—gaps and inefficiencies that had never been named. People were making the same decisions twice, doing work that could easily be automated, or improvising their own solutions to navigate unclear processes.

This is where product thinking shines: you're not gathering “requirements”—you’re observing user behavior, uncovering unmet needs, and spotting opportunities to simplify.

And just like with external products, what you learn here will change how you build.

You could even realise that the biggest win isn’t automation but clarity.

UX Matters - Even Internally

Internal products often don’t receive the same level of care as customer-facing ones. They rarely have good UX, and they’re expected to “just work.”

After all, your users are your colleagues. They’ll tolerate friction and adapt.

Maybe.

But if you believe quality and UX matter for external product adoption, why wouldn’t they matter just as much internally?

Internal users are still users. If a product is unclear, unreliable, or difficult to use, they’ll find workarounds—or stop using it altogether.

Yes, you should release fast. But speed isn’t an excuse for poor experience.
A buggy internal tool won’t cause churn—but it will damage trust.
And once that trust is gone, adoption stalls and motivation fades.

Here’s what you’d need to keep in mind:

• Design matters. Not because it needs to look pretty—but because clear, simple flows make a tool usable from day one.

• Early adoption is fragile. If the first release disappoints, re-engagement is 10× harder.

• Momentum compounds. Each small, reliable release builds confidence—and lowers resistance to change.

Treat your colleagues like users you need to motivate and inspire.

Because when they trust what you’re building, they won’t just use it, but improve it with you.

Simplify Before You Build

It’s tempting to custom-build internal tools that mirror every edge case and replicate all the complexity of current processes.

But many of those processes aren’t truly unique—they’re just undocumented, inconsistent, or unnecessarily complicated.

The goal isn’t to rebuild the mess in software. It’s to simplify and systematize the work behind it.

Start by asking: Is the problem the process—or the lack of one?

Before you write a single line of code, try this:

• Clarify the workflow: Can it be simplified or standardized before being digitized?

• Borrow what works: Lean on proven patterns and best practices instead of reinventing everything.

• Avoid overengineering: Don’t automate what you can simply eliminate.

The best internal tools aren’t the most complex—they’re the most clear and useful.

Momentum Is the Multiplier

The early stages of digitizing operations can feel painfully slow. You’re aligning stakeholders, untangling messy workflows, testing early versions, and constantly fixing what breaks.

But then something shifts.

Once internal users start seeing real improvements—faster approvals, smoother handovers, fewer manual steps—momentum begins to build.

And this changes everything.

Momentum builds trust. Trust drives adoption. And adoption creates pull instead of push.

Suddenly, teams are volunteering to be part of the next pilot. They’re suggesting their own use cases. Users are proactively sharing feedback. Your internal product shifts from being “just another tool” to becoming essential to how the business runs.

Staying patient is important—but you can also actively build momentum.

• Share progress: Highlight wins—time saved, errors reduced, increased adoption. These metrics fuel motivation and confidence.

• Be responsive: You should not build everything people ask for. But when you act on feedback, make it visible. “You asked for X—we delivered it this week.”

• Show what’s next: Don’t just demo what’s done. Share your vision for what’s coming. Paint a clear, realistic picture of how things will improve.

You don’t need perfection—just visible progress.

When users can see that progress and believe in the future you’re building, they become your champions.

Closing Thoughts

Digitizing internal ops is more than building a tool—it’s a rare opportunity to make your company run better.

You’re in an exceptional position: your users are close, your impact is direct, and the complexity is real.

Apply product thinking, and you won’t just ship a tool—you’ll build clarity across teams, surface what really needs fixing, and create momentum.

Let the product do what it does best: simplify, scale, and make your COO breathe a little easier.

Enjoyed this Article?

Subscribe to Lean Product Growth for regular updates on building and scaling a successful product organization. Insights, strategies, and actionable tips—delivered straight to your inbox.

Subscribe now

In the last product review meeting, the same concerning question came up: “Why is this product feature taking so long? Wasn’t it just a small software change that should’ve been wrapped up in a few hours? Why is the engineering team holding the next deal back?”

And while you might be easily pointing fingers at the engineering teams because of the missed deadline, the reality is far more nuanced.

The truth?

Software delays are rarely caused by laziness or lack of effort.

Instead, they’re the result of an entangled web of reasons that span across the whole organization. They are deeper, systemic issues that affect both developers and stakeholders.

Misaligned priorities, communication breakdowns, and technical debt all play a role, and until these challenges are addressed, the delays will continue.

Most Common Reasons Why Shipping Is Slower

Let’s dive into the most common obstacles behind slower feature delivery. Understanding these factors is the first step toward tackling them effectively.

The Illusion of Perfect Planning

Time estimates in software engineering are most often inaccurate.

Yet, as flawed as they may be, estimates are essential. They are the communication bridge between teams, stakeholders, and customers.

When timelines for the next product feature are estimated, several things commonly go wrong.

• Pressure to be optimistic. Engineers may feel compelled to give overly optimistic estimates to satisfy stakeholders and avoid disappointment. This “best-case scenario” approach often overlooks hidden complexities, leading to unrealistic expectations that later fall short.

• Overcompensation. On the flip side, some engineering teams go to the other extreme—adding excessive buffers to their timelines to confidently meet their promise. While this avoids disappointment, it slows overall progress and limits the team's ability to take on bigger challenges.

• Top-Down timelines. A worse scenario? Stakeholders set deadlines without consulting the engineering team. Sure, these timelines might be necessary to close the new deal, but without understanding the technical complexity, they create significant risks - disappointed customers or engineers being forced to implement "quick fixes" that damage the overall software health.

These mismatched expectations create a vicious cycle of frustration. Stakeholders feel let down by missed deadlines or slower progress. Engineering teams feel trapped between pressure to deliver quickly and the reality of their complex work.

Software Maintenance Matters

Software engineering isn’t just about development of new features—it’s also about keeping the existing codebase stable and functional.

In reality, a significant portion of a developer’s time is dedicated to maintaining the existing codebase. Bug fixes, upgrading outdated libraries, and adapting to evolving requirements aren’t optional—they’re essential to keeping the product functional and stable.

Beyond that, the process of launching a feature to production involves much more than just coding. Design, testing, deployments, and handling operational issues are all integral parts of the engineering workflow.

When this effort isn’t accounted for in planning, missing a critical deadline due to an urgent bug fix can be incredibly frustrating. However, rather than treating it as bad luck or using the bug as an excuse for the delay, maintenance work should be acknowledged as a fundamental part of the development process. It must not be overlooked or undervalued.

Software development isn’t a one-time effort. When estimating the cost of a new feature, don’t just plan for the development phase—factor in ongoing maintenance costs.

A good rule of thumb?

Expect to spend five times as much effort on maintenance over the lifetime of a feature as you do on building it.

The Domino Effect: Dependencies Among Teams

The product team is building a new feature. To ship it, they need the platform team to make changes in the SAP platform foundation. But the platform team can’t start because they’re waiting on a bug fix from an external vendor. And just like that, the planned timeline starts to collapse.

Here is the problem:

The product team creates their timeline based solely on their own effort, assuming the platform team will prioritize their request. But the platform team has its own list of urgent issues, and by the time they finally get to this “small update,” they hit yet another roadblock—a minor issue requiring action from an external vendor. The domino effect begins.

Dependencies between teams are one of the most frustrating and underestimated causes of delays. They create a fragile web of interconnected tasks, and the more complex the connections, the greater the risk of a cascading effect when even one link in the chain breaks.

What makes dependencies so problematic?

• Complex software and team architecture, inefficient processes. When the teams’ structure does not reflect the software system’s architecture, dependencies inevitably arise with every new feature development. Combined with weakly enforced processes or poor coordination, these seemingly minor dependencies can quickly snowball into significant delays.

• Wrong estimates and unmet commitments. If each team underestimates the true complexity of their work or downplay their dependencies on other teams, unrealistic commitments start to spread. When reality hits, timelines are blown, significantly extending the lead time for delivering the new feature.

• Diminishing urgency with each handoff. What begins as urgent for the product team gradually loses priority as it moves down the chain. For the next team, it’s just another task in the queue. The further the ball is passed, the less urgency remains—leaving a task critical for one team to stagnate in the backlog of another team.

Too Many Priorities, Not Enough Focus

I know how intense product review meetings can get.

Multiple stakeholders are anxiously waiting for their preferred feature to ship, each believing their initiative is the top priority. Or a business leader insisting that all the listed initiatives are of high priority—with a seemingly valid reason for each one.

As a result, the product and engineering teams chooses the route to work on all these “equally important” priorities in parallel.

The outcome?

Instead of speeding up delivery, everything ends up in the “In Progress” state and the lead time for any of the items to cross the finish line increases significantly.

While it might seem comforting for stakeholders to see all their preferred features marked as “In Progress,” the reality is, nobody wins.

Success isn’t about starting everything—it’s about finishing something. Until features cross the finish line, everyone remains stuck in a deadlock, waiting for progress that feels painfully slow.

Context Switching: The Hidden Productivity Killer

You’ve probably asked—or been asked— before: “Do you have five minutes for a small question?”

Sure, it might only take five minutes to address that small question, but the mental capacity you need to context switch and later return to your original task is a completely different story.

Software engineering, in particular, requires deep focus and sustained concentration. Each interruption—whether it’s a quick question, a meeting, or an urgent issue—disrupts this flow, impacting both the quality and productivity of the work. The more different the context of the interruption is from the task at hand, the greater the cognitive load required to refocus.

In fact, research shows that it takes 23 minutes to regain your mental state after being interrupted during deep work. This means even small, seemingly harmless interruptions come at a much larger cost than they appear.

But it doesn’t stop there. Small distractions and low-value meetings add up over time, further eroding productivity.

The impact becomes even more pronounced when engineers are juggling multiple priorities: fixing an urgent issue that pops up while working on a deep research activity has a high productivity cost.

This constant context switching not only slows progress but also increases the likelihood of mistakes and missed details.

Quick Fixes That Backfire Later

Engineers often face this decision to make: build a future-proof feature or deliver a partial solution - a quick fix necessary to meet the tight deadline.

With pressure arising and no time to spare, they’re forced to choose the shortcut, promising themselves they’ll revisit and fix it later.

But that "later" moment rarely comes.

As new features, deadlines, and priorities pile up, these shortcut improvements get buried in the backlog, forgotten until they show their ugly face at the worst possible moment - during an important go-live. The cost of that quick fix then resurfaces and technical debt now has to be paid off.

Shortcuts are a ticking time bomb in software development, and their true cost is always paid—just later, and often at a higher price.

Burdened By Legacy Decisions

A software system is built on decisions made in the past.

While these decisions were valid at the time, as business evolves, growth accelerates, and technology advances, those once-sound solutions can quickly become outdated. The architecture that effectively served the business yesterday becomes a bottleneck for the business today.

Modernization—whether through IT migrations, re-architecting systems, or adopting newer technologies—is a natural and necessary part of the software lifecycle.

Yet, when engineering teams bring these issues to the table, they’re often met with resistance.

Why?

The investment required for big changes is significant, and the benefits aren’t always immediately visible. For business leaders focused on securing the next deal or expanding market share, prioritizing a migration is a hard sell. After all, a migration won’t win the new contract, but it could delay one.

This lack of awareness about the long-term benefits leads to modernization efforts being deprioritized.

The result?

Teams continue to patch and work around outdated systems, creating inefficiencies and increasing the risk of failure. The business might feel like it’s saving time and money, but in reality, it’s gambling with the future software stability and team’s productivity.

To Sum Up

The reasons behind software delays are rarely straightforward. They stem from a web of interconnected challenges—ranging from organizational priorities and team structures to inefficient processes and the technical quality of the software itself.

This isn’t an engineering problem only.

It’s a shared responsibility. From company leadership setting realistic priorities, to stakeholders understanding the complexities, to product managers prioritising effectively, and engineers advocating for better practices—everyone has a role to play in addressing these issues.

In the next articles, I’ll dive into practical solutions and explore what each group—leaders, stakeholders, PMs, and engineers— can do to tackle these challenges and help ship features faster and more predictably.

Stay tuned!

Access other relevant content on Lean Product Growth

If you’ve spent more time in the software industry, you’ve likely encountered an IT migration at some point. Migrating systems can be one of the most challenging phases of product development, often filled with unexpected hurdles.

Unfortunately, the statistics aren’t encouraging. Over 80% of migration projects either exceed their budgets or end up being abandoned altogether. These figures emphasize the importance of having a solid plan and strategy in place to navigate the migration journey successfully.

IT migrations can take many forms, here are some typical examples:

• Data Migration involves transferring data from one storage location to another.

• Application Migration occurs when software needs to be refactored, rebuilt, or replaced entirely with a new system, often requiring also migration of data.

• Cloud Migration refers to shifting systems from traditional on-premise data centers to cloud environments or moving from one cloud provider to another.

This article covers:

➤ Examples of IT migrations and when do you need them?

➤ What makes an IT migration project so complicated?

➤ How can you navigate the risks to ensure a smoother transition?

When Do IT Migrations Become Necessary?

There are many reasons why software companies may face the need for migration. Let’s look at some of the most common scenarios:

Outdated Technology

Technology evolves rapidly. New, more efficient solutions enter the market, offering improved performance, better ROI, and lower maintenance costs. This evolution often necessitates moving away from legacy systems. In some cases, companies are forced to migrate to a newer technology because the one they rely on is no longer supported.

A good example is COBOL-based systems, which many businesses had to retire due to a shrinking talent pool, lack of support, or high mainframe costs.

Another example is the shift from on-premise systems to cloud applications, as cloud solutions provide greater flexibility and scalability and have become the new norm in the market.

Scalability Limitations

Another very common driver for migration in growing companies is the inability of current systems to scale with business growth. Often, architectural or technology decisions made during the early stages of a company’s development—while suitable at the time—become bottlenecks as the user base expands and business needs evolve.

It’s a harsh reality, but at some point, investing in an upgraded, scalable solution becomes inevitable. Otherwise you can’t keep pace with future business demands.

Mergers and Acquisitions

Mergers and acquisitions often create an urgent need for migration. Companies must consolidate multiple systems into a unified platform to streamline operations effectively.

The primary drivers for this consolidation include reduced maintenance costs, improved scalability, and enhanced return on investment (ROI).

Accumulated Technical Debt

Migration due to accumulated technical debt is not something to be proud of. However, if you haven’t kept an eye on continuously optimizing the quality of your product, rebuilding becomes the most viable option at some point.

When teams rush to meet deadlines or prioritize short-term deliverables, corners are often cut in code quality, architecture, or documentation. Eventually, companies reach a point where a tough decision must be made: either refactor or entirely rebuild the platform to regain efficiency and lay a solid foundation for future growth.

The Compelling Benefits of Migration

Whether migration is a necessary transition or a strategic choice, the advantages are clear and substantial.

A successful IT migration can unlock significant financial and operational benefits:

• Enhanced development productivity

• Decreased maintenance costs

• Accelerated development cycles

• Improved scalability

• Stronger positioning to meet evolving business needs

The Biggest Challenges of an IT Migration

While the benefits of IT migrations are real and appealing, the reality often strays far from the plans laid out on paper.

Why is this the case?

What are the most significant challenges companies face when tackling an IT migration?

Legacy Systems as Revenue Drivers

The current systems may be outdated and costly to maintain, driving the need for migration.

However, these "legacy" solutions often form the backbone of revenue generation. Typically, they are stable and reliable, having undergone years of stability improvements.

Replacing them too quickly or without sufficient planning can risk essential business operations, leading to potential revenue loss and customer dissatisfaction.

Knowledge Gaps and Unsupported Technology

Migrating a system effectively requires a deep understanding of the current solution.

In many cases, the developers who have built the software are no longer available, and documentation is sparse or outdated. This forms a lot of critical knowledge gaps.

Without access to the expertise needed to interpret or rebuild these systems, teams can struggle to understand how they function, making the migration complex and prone to errors.

Resistance to Feature Deprioritization

Imagine the following scenario: multiple company divisions are running their own platform, each tailored to specific customer needs. To reduce maintenance costs and enable scalability, a migration project is launched to unify these platforms, migrating all customers to a single, centralized system.

The new system must meet the diverse requirements of each division, but ensuring full functional compatibility can be extremely challenging.

This approach not only places a heavy burden on the migration project, but also makes the new system highly complex and difficult to maintain. Deprioritizing certain features and finding the right balance is essential, but it can generate substantial resistance from stakeholders who fear the unified system won’t fully meet their needs.

The Need of Extensive Testing and Iteration

Legacy systems have achieved stability over time. The new system will still require rigorous testing and refinement to reach an acceptable level of reliability. Underestimating the time needed for testing and rushing to go to production to soon can lead to disruptions and unexpected bugs.

With many customers already using the current platform, there’s little tolerance for issues. Whether the migration is invisible to them or changes their experience, the only acceptable outcome is a system that performs as good as, or better than, the old one.

The Human Challenges of Migration

Migration is not just a technical challenge—it can significantly impact people and can pose considerable obstacles.

Employees workflows and customer interactions are often disrupted. Employees need training on new processes, and customers may need to adapt to different ways of engaging with your product or services.

Without a comprehensive change management strategy that includes training, clear communication, and a gradual transition to ease adjustment, the risk of a failed migration is too high.

Maintaining Dual Platforms During Transition

During the migration, you need to maintain both the old and new platforms. This duality can be resource-intensive and costly.

Next to that, customers expect continued improvements to the legacy system even as the migration is underway.

Deciding how much to invest in the old platform versus focusing on the new system becomes a strategic balancing act, adding pressure on resources and timelines.

A Realistic Approach to IT Migration

Successfully managing an IT migration requires a realistic strategy that accounts for its complexities. Here are some practical strategies to consider:

Plan Realistically

A migration project is rarely a quick fix. Depending on the complexity of your systems and business needs, it will likely take more than a year—not weeks.

Treat migration as a complex project requiring dedicated time and attention. Be prepared for unexpected challenges, and set clear expectations early to avoid unnecessary frustrations.

Dedicate Migration Teams

Managing both the current platform and a migration simultaneously is resource-intensive and often unsustainable. It is unrealistic to rely on a single team for both.

Instead, consider setting up dedicated migration teams who can focus fully on the migration project.

You might also choose to outsource specific tasks, like data migration, to specialists. A data migration specialist would bring expertise in anticipating potential complexities and would follow a robust process to ensure high data accuracy, making the migration more efficient and reliable.

Does the migration impact operational workflows or directly affect customers? Be sure to include these considerations in the migration roadmap. If the scope is significant, it’s worth dedicating a team specifically to manage change. As this is essential prerequisite of a successful migration.

Migrate in Phases: A Cohort-Based Approach

Phased migration is one of the most effective ways to reduce risk.

When migrating customer data, consider a cohort-based approach. Cohorts are groups of customers segmented by specific criteria, such as feature usage, migration ROI, or migration risk level. For example, you might start by migrating new customers to the new platform, followed by those using only core features, and finally, customers with more complex or customized needs.

This step-by-step approach allows for gradual learning and adaptation. It gives you time to identify and resolve issues early, while minimizing disruption and risk for both customers and your business.

Resist the Urge to Improve the Old System

One of the difficult decisions during a migration is whether to continue enhancing the old system while transitioning to the new one.

Unless there is a compelling return on investment or a significant risk in leaving the old system as is, avoid investing resources in it. Focus your efforts on the new platform instead.

This will reduce the time it takes to deliver the new system. It will also ensure that any effort spent on improvements has long-term benefits.

Be Prepared to Lose Some Customers

During the migration, there will likely be features that some existing customers won’t have immediate access to.

Are you prepared to say "no" to certain customers, knowing there is a potential risk of churn? If you are not ready for this, you may need to reconsider whether your migration strategy will truly deliver the expected benefits.

Avoid falling into the trap of making the process too complex in an attempt to please everyone. This can drain resources, delay the migration project and make the product overly complex to maintain on long term.

Final thoughts

IT migrations are never simple. They involve technical challenges, change management, and detailed planning.

AI can help make parts of this process much easier. By automating tasks like data mapping and error detection, there is a lot of potential to reduce errors and save time.

However, significant challenges will still remain—particularly in managing expectations, ensuring system compatibility, maintaining operational stability, and adjusting workflows for employees and customers.

Avoid the temptation to rush your next migration project. Stay alert to the complexities that may arise, and regularly reassess the feasibility of your plan to ensure a successful transition.

Access other relevant content on Lean Product Growth

Today’s newsletter is about architecture principles.

• What are architectural principles and why are they essential?

• How to define these principles?

• What are some examples of common principles that promote scalability?

When designing and implementing a software system, numerous decisions must be made. Where the team is small, these decisions may be handled or overviewed by a single individual. The need for principles may not be immediately apparent.

However, as the software and company grow, multiple stakeholders become involved in making daily design choices.

Without following any principles, software can become complex, its structure inconsistent and confusing, technical debt increases with irreversible decisions that hinder future progress.

Architecture principles serve as guiding boundaries, providing direction for the software design decisions. They are high-level rules that should be adhered to, ensuring that the system remains cohesive and aligned with business needs.

How to define architecture principles

Think about the value statements displayed in a company's office. While creating a list of principles is essential, true success lies in embedding these principles into the organization's culture and guiding people's behavior based on them.

The same applies to architectural principles. Drafting a document with a list of principles that nobody pays attention to is simply a waste of time. Success lies in building a culture where the technical team feels connected to and empowered by these principles.

Start with the goals of the product

Principles serve as a bridge between the overarching vision of a product and its practical implementation. They are derived from the primary goals.

Consider the goal tree example depicted below.

Read more

Scalable architecture is foundational for business growth. It's not merely a technical decision but rather a strategic imperative essential for supporting growth and gaining a competitive edge.

In the previous article, we delved into scalability from a functional standpoint, examining the system’s flexibility to evolve in terms of functionality without the need for significant reengineering.

This article shifts the focus to scalability from a performance perspective, often referred to as Performance Scalability, or simply Scalability.

What is Performance Scalability

Performance Scalability refers to the ability of a system to handle increasing traffic (Load Scalability) and data volumes (Data Scalability) without compromising its responsiveness or speed. In this context, Scalability ensures that the system can maintain its speed and responsiveness as demand grows over time.

As the business expands, the number of users placing load on the system grows as well. This creates pressure to the extent that the initially allocated resources become insufficient to handle the increased load. Consequently, the system needs to "scale," necessitating additional resources to accommodate the increased load.

Scaling can take two approaches.

Vertical scaling, also known as scaling up, involves enhancing the resources of a single server or machine to manage higher loads. This may include upgrading the CPU, memory, storage, or other components to boost performance and capacity. However, this approach has limitations in terms of scalability and cost-effectiveness.

Alternatively, horizontal scaling, also referred to as scaling out, entails expanding the system by adding more servers and distributing the workload across them.

Scalability vs Elasticity

A special type of scalability is elasticity. Although similar in concept, these terms have distinct meanings.

Scalability refers to the system's ability to manage gradual increases in traffic and data as the number of users grows. As the load gradually increases over time, the ability of the system to scale only manually is often sufficient.

However, there are instances where gradual scalability is not sufficient, and the system requires support for elasticity. Elasticity refers to the system’s capacity to handle sudden peaks in load and swiftly return to normal behavior afterward.

To achieve elasticity, the system must be capable of automatically scaling servers based on the current load. Alternatively, dedicating excessive capacity upfront is an option, but this is of course not desirable, as it leads to underutilization during periods of low demand and therefore unnecessary high costs.

A typical example illustrating elasticity is a platform for purchasing tickets to a newly released movie. When ticket sales open, there is a significant surge in traffic as everyone attempts to buy tickets before they are sold out.

An elastic system would seamlessly manage this increased traffic without any disruption. However, a system lacking scalability and elasticity will struggle to cope with the new situation. This could result in broken system performance or even make the system completely unavailable during peak periods.

Scalable Architecture is a Business Choice

Scalable architecture is often seen solely through a technical lens. Yet it's crucial to recognise that this is a strategic decision. Investing in a scalable architecture should be driven by clear business needs and aligned with broader organisational objectives.

The benefits of this architecture include:

Cost-effective growth: Cost is the primary driver for scalable architecture. While investing in expensive migration to scalable architecture might not be justified for businesses with stagnant growth, it is crucial for growing businesses anticipating an increased customer base. Scaling the architecture becomes a pivotal asset and a key enabler for growth.

Agility: Scalable architecture (both functional and performance scalability) brings a significant benefit in agility. A company supported by an agile software landscape has the advantage of swiftly adapting to evolving business needs, whether it involves entering new markets, introducing new products or services, or adjusting to changing customer preferences.

Resilience: Businesses supported by a scalable foundational architecture can maintain uninterrupted operations even amidst unforeseen challenges such as sudden spikes in traffic, hardware failures, or cyber threats. This not only reduces operational costs and headaches but also enhances the user experience.

Competitive advantage: Scalable architecture provide a significant competitive advantage. By fostering agility and adaptability to new market conditions and enhancing user experience, companies supported by scalable foundations can outpace competitors.

How to Achieve Scalable Architecture

There is no simple blueprint for building a scalable architecture. It demands expertise, a scalability mindset, and an understanding of the business. Let's review the key principles to consider when aiming for scalable architecture.

Understand your scalability needs

Designing an architecture involves striking a careful balance of trade-offs and considering various "what if" scenarios. Understanding the business goals is essential for this task.

Questions such as who the end customers are, their expectations, the projected number of users, and potential growth are crucial. Additionally, identifying peak periods of activity, assessing the business impact of system failures, and determining the need for handling confidential data are key considerations.

The defined business goals drive the architecture principles and guidelines for implementation. These aspects should not only exist on paper but should be deeply integrated into the product organization. Designing a system that requires 99.999% uptime differs significantly from one where occasional downtime may be tolerated without significant impact.

Embrace decoupling from the start: transition to services when ready

Large monolithic applications are difficult and expensive to scale due to their single deployable unit nature. Scaling such applications requires scaling the entire system's functionality, resulting in rapidly escalating costs.

A fundamental principle for achieving scalability is to break down a large codebase into services. These smaller deployable units can be maintained, deployed, or scaled independently. This approach is known as service-oriented architecture (SOA) or micro-services architecture when the services are more granular and there is also separation of data.

To promote high scalability of a larger system, this pattern is inevitable. However, moving to micro-services architecture must be approached carefully, as it has its own downsides.

• The shift from in-memory to network-based communication between services introduces latency and performance concerns. Imagine substituting an existing call to a component in a loop with a network call; this could cause substantial performance degradation and potentially lead to system failure.

• Micro-services often depend on each other; a failure in one service can trigger cascading failures. System success depends on how well the system is designed for resilience and the failover mechanisms implemented.

• Security poses a greater challenge with micro-services as they expose individual APIs. Establishing security measures in the distributed environments becomes critically important.

Maintaining and managing micro-services is inherently more complex compared to monolithic applications, yet when executed effectively, it offers significant advantages.

Transitioning directly to micro-services is not recommended. Instead, it is advisable to begin with a modular architecture and decoupled components. This approach ensures that as the codebase expands, transitioning to separate services becomes much more manageable.

Coding best practices can make a significant difference

Ensure that best practices are applied during coding to optimize application performance before considering scaling. This involves implementing efficient algorithms and adhering to coding guidelines. For instance, minimizing the use of complex nested loops, leveraging caching mechanisms to reduce traffic between components, and optimizing communication between distributed components can significantly enhance application efficiency.

Developing with a performance-oriented mindset can yield substantial improvements in application performance. While these optimizations have their limitations and scaling at a later stage would still be necessary, they can mitigate the need for unnecessary early scaling, ultimately resulting in cost savings for the business.

Stateless services

A fundamental principle of scalable architecture is the use of stateless services, essential for enabling horizontal scaling. The concept revolves around ensuring that each request to a service is self-contained and independent of past interactions (state). This approach simplifies the distribution of incoming traffic across multiple service instances.

Asynchronous communication

Another crucial principle is minimizing synchronous communication between services. Asynchronous communication decouples system components, enabling independent operation without reliance on one another to complete tasks. Implementing asynchronous communication patterns enhances overall system performance, system responsiveness, and resource utilization.

Utilize cloud services

Cloud platforms provide elastic resources, auto-scaling capabilities, and managed services, simplifying scalability. This can yield significant benefits, positively impacting costs and scalability.

However, migrating to the cloud doesn't automatically resolve all issues. Many organizations are embracing this trend, only to realize that simply lifting their applications to the cloud without proper preparation does not yield the expected benefits. While the cloud offers great potential, applications need to be cloud-native and need to follow the practices mentioned above to fully leverage its advantages.

Monitor, analyze and optimize

Monitoring is essential for achieving optimal system performance. Utilizing monitoring and analytics tools can provide valuable insights into system performance, scalability, and resource utilization. This becomes particularly important in a micro-services setup, where identifying and resolving slowdowns within the network of services is essential for maintaining overall system efficiency.

Building a Scalable Architecture in an Agile World

Making the right architecture choices is crucial for creating a future-proof system that can adapt and grow alongside your business needs. However, this can be challenging in today's agile environment where the future is unpredictable.

According to Gartner's report on "From Fragile to Agile Software Architecture" (2019), traditional architectural practices often clash with modern agile software development methodologies, leaving teams without clear guidance on how to architect their systems effectively.

It's important to recognize that building great architectures is a gradual process that requires continuous iteration, feedback, and adjustment. Take, for example, companies like Uber or Spotify, known for their flexible microservices architecture. These architectures evolved over time through iterative development and refinement.

As Adam Gluck from Uber Engineering puts it,

"Evolving a microservice architecture is more like 'trimming a hedge'—a dynamic and progressive process—rather than a one-time, top-down effort."

This blog is part of the series on Scalable Software Architecture. Below, you'll find an index of the articles in this series:

1. The Software Development Reality: The Need for Scalable Architecture

2. The Role of Flexibility in Scalable Architecture

3. Empowering Growth: The Importance of Performant Scalable Architecture

4. How to Establish Architecture Principles for Successful Scaling

Subscribe to receive the latest insights.

Subscribe now

In my previous article, I discussed why adopting a scalable architecture is necessary to enhance business acceleration. In this article I will delve deeper into the concept of scalable software architecture and in particular Flexibility as one important aspect of a scalable architecture.

Flexibility - one angle of Software Architecture Scalability

Scalable software architecture is designed to expand seamlessly with the growth of a business. It's agile, adaptable, and has the capacity to accommodate dynamic changes.

When discussing scalable architecture, there are multiple angles to consider:

Functional scalability, often referred to as Flexibility, is one aspect of software scalability. It highlights a system's ability to adjust to changing requirements, allowing for the addition of features, fixing bugs, or updating the system without significant effort or requiring a complete redesign.

Load scalability views scalability from a different perspective. It's about the system's ability to handle increased traffic without compromising performance efficiency. A scalable system can manage high-peak traffic, ensuring reliable and high-performance operation at optimal costs.

Data scalability is similar to load scalability, but the focus here is on the system's efficiency in managing growing volumes of data. It becomes crucial how optimally the software processes, stores, archives, or integrates scalable data storage solutions to maintain optimal performance.

What do we expect from a Flexible Software Architecture

If a flexible architecture enables fast and effective implementation of system changes into production, it should embody a high degree of:

1. Maintainability - Simplifying updates and issue resolution within the system.

2. Testability - Streamlining the testing process post-update implementation.

3. Deployability - Ensuring rapid deployment of tested changes into production.

These three elements are crucial stages in the process of implementing changes in a system and transitioning it to production. A flexible architecture should support each of these areas.

How to achieve Flexible Architecture

So, what principles do we need to follow to build an architecture that is flexible and supports easy maintenance, testing, and deployment?

1. Build the system in well-defined modules

At the heart of a Flexible Architecture lies Modularity.

Modularity entails a software system characterized by a clearly defined structure. It involves constructing software using smaller, well-defined modules (or components) that have minimal interaction with each other. These modules adhere to the same modular approach internally, consisting of submodules with an optimal number of dependencies.

An analogy for modular architecture is a library, where bookshelves represent modules. In a well-organised library, each bookshelf serves a specific genre, easily managed independently. Additionally, each bookshelf is further divided into categories based on titles, making it effortless for visitors to find a specific book. This stands in stark contrast to a disorganised library, where finding the series of your favourite books can be nearly impossible, and chaos ensues when multiple people search simultaneously.

There are a few key principles of a modular architecture:

Principle 1: Single responsibility modules. Each module is dedicated to a specific function; it should have a single responsibility. When a module's purpose isn't clearly defined, it tends to accumulate unrelated logoc over time, growing into a large module that becomes a maintenance bottleneck.

Principle 2: Clear interface hides implementation details. Modules maintain well-defined interfaces for communicating with other components. This ensures that a module's code interacts solely through its interface, encapsulating the implementation details. In the figure above, Module A communicates with the external system through a dedicated submodule. This setup enables seamless replacement of the external system by updating only the relevant submodule, minimising disruptions in the rest of the system.

Principle 3: Minimising dependencies. Modular design aims to minimise connections between components. Fewer dependencies lead to decoupled modules, making it easier to replace one with less impact on the system. This emphasis on minimising dependencies is particularly vital for higher-level modules, although lower-level submodules may accommodate more intricate connections.

Modular architecture facilitates Maintainability

In a modular system, implementing small updates becomes easier, as developers can quickly pinpoint where a change should be made within the well-defined modules. The fix remains encapsulated, preventing changes from affecting the entire system.

Expanding the system with new functionality is also manageable within the well-defined modular system framework, where each component has its designated place.

Additionally, replacing a component is more manageable compared to a non-modular system because it only affects a small portion of the codebase. This flexibility enables seamless changes to the system.

2. From Modules to Deployable Units

Before we proceed further, let's delve into the concept of a deployable unit and monolithic architecture.

A deployable unit refers to a single piece of executable code.

In a monolithic architecture, the entire system is contained within a single deployable unit, commonly known as a monolith.

While discussing modularity, we didn't explicitly mention that modules are independently executable units. Consequently, a system with a modular architecture can still be a monolith - a modular monolith. In such cases, the entire system functions as a single executable component, with a well-organised codebase structured into software modules.

A monolithic architecture is simple and straightforward, making it a suitable choice for smaller systems.

However, it does come with limitations. As the system and the team responsible for its maintenance grow in size, sustaining it as a monolith becomes progressively challenging.

Smaller deployable units enhance Deployability

Monolithic architecture presents limitations in deployability. Even minor modifications in the system necessitate deploying the entire system, posing obstacles for the entire team. This leads to code freezes, lengthy and intricate testing procedures, and prolonged deployment cycles spanning weeks.

The concept of frequent deployments becomes unfeasible, and introducing new changes for customers on a regular basis becomes both risky and unfavorable for the business. This often results in delayed implementation of new features or bug fixes, leaving them in production for weeks.

To address these challenges, segmenting the application into distinct deployable components (or services) proves advantageous.

Applications with smaller deployable units experience quicker time to market and increased agility. Deployment processes are shorter, risks are smaller, and customers can benefit from more frequent code updates.

Smaller deployable units promotes Testability

Large monolithic applications also add complexity to the testing process. Even if the monolithic application has a comprehensive suite of regression tests, implementing a change and redeploying the entire application entails executing an extensive list of tests, making the process slow and often frustrating for the engineering team.

Partitioning the monolith into smaller deployable units, when done correctly, can significantly enhance testability. When a change is made only in a smaller deployable component, the testing scope becomes more focused, and testing becomes faster. Additionally, maintaining unit tests for smaller deployable units is considerably easier.

How to approach the decomposition process

However, the process of decomposing a monolithic structure into separate components must be approached with careful consideration and a clear rationale.

Companies often aspire to transition to a micro-services architecture (where the system is divided into very small deployable units, each with its dedicated data storage), expecting direct benefits. However, if not executed correctly, this intricate renovation process can significantly increase the existing challenges.

The most daunting scenarios arise when attempting to prematurely separate small services from a large monolith. Not only is the separation itself challenging, but replacing software code dependencies within the monolith with (network) dependencies between deployable units can significantly impact the system's performance efficiency.

To transition from a monolithic to a service-oriented architecture, the initial step is to modularise the architecture and then gradually isolate the most independent modules as deployable units.

Final thoughts

The path toward scalable and flexible software architecture is a strategic imperative for businesses pursuing agility and growth.

It's essential to recognise the trade-offs and challenges that are part of this journey. Software architecture is defined by these trade-offs. It's crucial to carefully weigh the pros and cons and find the right balance. Achieving perfection in every aspect is impossible.

This blog is part of the series on Scalable Software Architecture. Below, you'll find an index of the articles in this series:

1. The Software Development Reality: The Need for Scalable Architecture

2. The Role of Flexibility in Scalable Architecture

3. Empowering Growth: The Importance of Performant Scalable Architecture

4. How to Establish Architecture Principles for Successful Scaling

Subscribe to receive the latest insights.

Subscribe now

At TechWavve headquarters, a new announcement has brought excitement—a green light for the upcoming merger. New exciting opportunities, substantial growth, a doubled customer base, and fresh market openings are on the horizon. This is going to be a game-changer.

During this executive team's excitement, a big cloud of concern over the head of engineering. Immersed in the ongoing complex projects, the engineering team cannot even comprehend the scale of the challenge ahead. Merging or transitioning the already complex software landscape with the software from another company is a formidable task, like scaling Everest.

What hinders the software landscape from supporting the desired business agility?

The complex reality of software development

In today's era of software development, numerous tools, technologies, and programming languages are available, simplifying development and making it accessible to a wider audience. With this modern technology, there is a prevailing expectation that the entire product development process should be more straightforward and efficient.

However, the reality for many companies doesn't quite match this expectation of effortlessly building software applications. Why is that?

When we look back at product development from a decade ago and compare it to the present, we must acknowledge significant improvements. Systems can now be built more swiftly, with better product quality and an incomparable user experience.

But despite these advancements, fundamental forces persist, making software development more complex than we expect.

Software development is more about maintenance than development

A seemingly minor addition to meet a customer's request involved just 10 new lines of code. However, this new code is expected to function reliably for the next 5, 10, or even more years, as long as it's in use by any customer in a production environment. Consequently, sustained maintenance of this code is necessary during these years, requiring a team member who thoroughly understands its functionality.

Software maintenance consists of post-release activities, including updates after the code has been released to production. This can involve addressing outdated features, improving performance, fixing bugs, or adapting to evolving technologies.

Unfortunately, the significance of code maintenance is often overlooked. Despite the belief that code is a one-time creation, the reality is that, at some point, this code will either malfunction or, at the very least, increase the cognitive load for the team.

Statistical data shows that software maintenance accounts for 80%-90% of the total software development effort. On average, on a yearly basis, software maintenance costs may range from 15% to 25% of the initial development cost. In simpler terms, to build an application and keep it live in production for at least 5 years, you should anticipate dedicating five times more effort (and cost) than initially planned for development.

Size and complexity matter

Each line of code introduced into a system necessitates ongoing maintenance. Consequently, the size and complexity of the software significantly influence the productivity of development.

That’s why creating a new system from scratch is typically a swift process. During the initial development phase, the focus is solely on construction, free from the weight of past code created by someone else. Developers are also enthusiastic and motivated to introduce a brand-new innovative feature.

However, the scenario shifts as the system expands, and the once-new code ages into what is commonly known as legacy code. The complexity of the legacy code is further compounded by frequent changes, making it less organised and understandable when new developers join the team.

Despite the team's eagerness to progress with fresh new features, the challenges of correcting issues from past code pull them back into the realm of addressing and maintaining the existing system.

Research has explored the relationship between system size and the effort needed for maintenance. The simplified formula within the COCOMO model is:

Effort = a * (Size)^b

where, "a" represents the productivity of the development team, and "b" indicates the scalability of the product.

Without delving into complex mathematics, it's clear that size significantly impacts effort. And even more, the more complex the software architecture, the steeper the growth trajectory in terms of required effort.

Software development beyond software programming

Software development goes beyond mere programming. Typically, the software implementation constitutes less than 50% of the total time invested in bringing the code to production.

The entire software delivery process involves more than just coding—it includes design, code testing, deployments, and handling operational issues. Maturing this processes and automation can significantly reduce the effort required for these supporting activities. However, achieving these improvements requires allocating time for technical enhancements, which is often challenging alongside the demand for new features.

Adding to this complexity, bringing a feature to production involves addressing new compliance and security requirements, aligning with stakeholders, and accommodating inevitable changes in requirements. Over time, the cumulative impact of these factors intensifies, especially as the system expands.

Business agility through scalable software architecture

In today's dynamic business landscape marked by swift shifts, mergers, and heightened competition, software systems demand agility. The speed of software development hinges on key factors – size and scalability. While optimising size through lean features is important, the natural growth of a system over time is inevitable.

Crucially, the scalability and complexity of a software system, rooted in its architecture, significantly impact agility. A large, non-modular system with complex code obstructs new feature additions and timely bug fixes. A monolithic structure managed by multiple teams cannot support releasing new features fast. The downsides of such architecture become clear as data volumes and incoming requests increase, leading to performance issues and rising scaling costs.

This blog is part of the series on Scalable Software Architecture. Below, you'll find an index of the articles in this series:

1. The Software Development Reality: The Need for Scalable Architecture

2. The Role of Flexibility in Scalable Architecture

3. Empowering Growth: The Importance of Performant Scalable Architecture

4. How to Establish Architecture Principles for Successful Scaling

Software development is costly.

Introducing a seemingly high-value new feature to your product can present substantial challenges that test the mettle of your development teams.

Consider the time investments necessary for feature development, the inevitable troubleshooting of unforeseen issues (often surpassing initial estimates), and the frequently underestimated aspect of ongoing code maintenance.

Data indicates that an astonishing 70-80% of the total lifecycle costs of a software application are attributed to maintenance alone. This statistical insight underscores the importance of an often-underestimated aspect: ongoing code maintenance.

What's more, when code is delivered without a keen focus on quality, it worsens the problem significantly, substantially increasing the complexity of your teams' work. This leads to a considerable accumulation of technical debt within the codebase.

At some point, a pivotal decision must be made: continue to manage the existing code, undertake a comprehensive code refactoring project, or embark on a complete rebuild of the product?

How Did We Accumulate So Much Technical Debt?

Technical debt is a familiar concept in software development. It emerges when developers take shortcuts or compromise to meet pressing project deadlines or immediate deliverables. 

Every product's codebase naturally collects some technical debt, which is normal and not necessarily a problem. However, issues arise when this debt keeps piling up over time and slows down the velocity of software development. This slowdown reorients the focus from innovation and feature enhancement to issue resolution. 

This shift means that a disproportionate amount of developmental bandwidth is allocated towards 'firefighting' rather than advancing the product roadmap.

So, what contributes to the accumulation of technical debt within a software development lifecycle? 

Foundational Decisions

Just as a solid foundation is crucial when building a house, the decisions made during the initial phases of software development have a lasting impact on the project's trajectory. Choices concerning software architecture or technology selection are strategic imperatives. If these decisions aren't made correctly from the start, they might lead to substantial challenges as the project evolves.

Initial architectural decisions may not exhibit immediate drawbacks, especially in smaller teams.  However, as the scope of the project broadens and the size of the development teams scales, these initial decisions can gradually evolve into primary bottlenecks. They may inhibit workflow efficiency, obstruct seamless communication, and escalate operational costs. Eventually, it can reach a point where releasing even a simple software feature becomes a substantial challenge and a source of significant frustration for the product teams.

Poor Approach Towards Quality

A lack of commitment to clean and quality code is dangerous.

Similar to the regular maintenance required to preserve the value of a sturdy house, software quality demands continual attention. A lapse in this commitment can convert software from a strategic asset to a burgeoning liability.

Some product teams may not fully grasp the consequences of subpar quality. They lack quality assurance mechanisms or best practice quality frameworks integrated into their operational procedures.

However, more commonly, teams recognise the implications of low quality but often lack the authority to prioritise quality improvements over the pressing need to deliver the next set of features. They work under the pressure of impending deadlines, which makes it challenging to shift their focus towards enhancing quality. There's often the false optimism that the following month will provide the bandwidth for quality improvement initiatives, yet more often than not, things get even worse.

These scenarios often highlight a disconnect between the "business" and "technology" aspects, where technology teams tend to follow the business directives. While business stakeholders understandably pursue feature expansion as a strategic goal, such a focus often deviates from the path of investing in the reduction of technical debt.

Desire For More Features

New features are exciting. They enhance the product, enrich user engagement, and increase its core value.

But did you know that every new feature, regardless of its code footprint, carries associated and often underestimated overheads? It entails added efforts for testing, maintenance, and bug fixing. And not to ignore, it increases the overall complexity of the product and demands more mental effort from the product team to maintain the entire codebase.

Think of it like buying another coffee machine. While it enhances your daily routine, it also requires regular maintenance to operate at peak performance. And if you make the wrong choice, or if you fail to maintain it regularly, it becomes more clutter than a useful addition to your household.

Strategic Decision-Making Amidst Mounting Technical Debt

When you are at the stage when there is a lot of debt accumulated in your codebase, you find yourself at a critical decision-making vertex. The weight of each choice you make is inescapable, carrying significant implications for both your product teams and broader business objectives. 

So, what strategic options should you consider?

Option 1: Maintain The Status Quo

You have the option to retain the current situation. This may translate to zero immediate financial or time investment. While this route may appear cost-effective on the surface, it carries the risk of stagnation or even deterioration as your project or team scales.

You might identify some low-hanging fruit - easy-to-fix issues. While addressing these issues can bring some positive changes, the fundamental problem will likely persist, and the overall impact may be limited.

Retaining the status quo can be a strategic move under specific circumstances, such as when your product is approaching the end of its lifecycle or requires minimal ongoing maintenance, remaining stable with infrequent updates. Alternatively, it can be a suitable option when you're nearing a significant milestone, like a new investment round or a major product launch. In these instances, delaying decisive action on underlying issues could be justifiable as a short-term strategy.

Option 2. Prioritise Refactoring Over New Development

Another scenario involves code refactoring, which means renovating and improving the codebase.

This approach works well when executed cautiously. Without the discipline to delay or slow down the pace of new feature development, extensive refactoring can lead to unfavorable outcomes. It carries inherent risks, and if not carried out with care, it can introduce numerous new issues into the product.

The benefit of this approach is that, if successful, it leads to an enhanced codebase. It also offers the flexibility to choose which parts of the codebase to refactor, eliminating the need for a complete rebuild.

This option is most appropriate when you've identified a specific foundational issue that doesn't require a full codebase rebuild. For example, it could involve breaking down a large component or resolving a performance bottleneck at the interface level.

Before starting any renovation initiative, conduct an initial assessment of the effort and associated risks. Sometimes, refactoring a single component of the software can have a profound impact on the entire product. Refactoring a database, for instance, can have ripple effects across the entire product, even if it initially appears to affect only one part. In fact, the more modular and higher quality the code, the smoother the refactoring process becomes.

Option 3: Parallel Rebuild And Migration

It might seem overwhelming, but when dealing with a product of exceptionally low quality, opting for a complete rebuild is often the most practical solution. In fact, it can frequently be a simpler and cleaner choice than extensive refactoring.

While it does require an investment, projecting the costs of rebuilding and maintaining a high-quality solution over a five-year period typically yields a more favorable outcome compared to persisting with and maintaining a low-quality codebase or refactoring it.

The positive aspect of this approach is that building a new product can occur in parallel with ongoing development. This can involve a dedicated team focused on the new product, which is a much cleaner and more productive approach. It presents the opportunity to make the right architectural decisions, leverage the most suitable technological stacks, and modernize the product.

However, it's essential to remember that once the new product is ready, the migration process comes into play. Be prepared: migrations are substantial projects on their own and require a serious commitment.

Strategic Imperatives For Mitigating Technical Debt 

None of the options listed above are straightforward; they bring risks and financial commitments and may decrease the team morale. After all, who would naturally gravitate towards the difficult task of refactoring over developing an innovative and shiny new feature?

The most effective strategy lies in preventive action—averting a scenario where such a complex decision becomes necessary. Here’s how you can do that:

Quality Awareness

Understanding that a low-quality product can have adverse effects on your business is the essential starting point.

However, ensuring a high-quality product is not solely the responsibility of the technology teams; it should be a company-wide mindset. Cultivate a culture where your product managers and stakeholders prioritize quality while pursuing new developments without compromising it.

Resisting the urge to cut corners in the face of pressing deadlines is undeniable. Remember, shortcuts must be the exception rather than the rule. Frequent reliance on shortcuts might bring exponential costs over time.

Support your product development team with the tools and best practices to seamlessly integrate quality into their work. Doing so elevates the standard of excellence consistently across your product and technological offerings.

Thoughtful Architectural Decisions

Unwise architectural choices have the potential to undermine the success of your product, even when you have a team of top-tier developers producing high-quality code. The foundational structure of a product is not easily altered in the future.

When setting up the fundamental architecture of your product, it's crucial to approach this task with great care, weighing the advantages and disadvantages of different options. Rather than focusing solely on the current state, consider the product's future growth. Opt for solutions that can scale and adapt as your product expands. Ensure comprehensive documentation of your decisions and the rationale behind them.

While it's impossible to guarantee a completely future-proof solution, making well-informed choices significantly reduces the risk of costly errors.

Maintain A Lean Approach

Approach product development with a lean mindset. A product free from unnecessary clutter is easier to manage, visually appealing, and more user-friendly.

Resist the temptation to introduce a new feature solely because it seems like a good opportunity or because one of your customers has asked for that.

Instead, aim for a balance, focusing on truly essential features. Prior to incorporating new features, ensure their necessity is supported by data. Embrace experimentation and validation before committing the product development team to a two-month project, followed by three times the effort for ongoing maintenance.

This lean-centric approach should be embraced on every level of the organization—from executive strategy-making and sales initiatives to product management's feature-value assessments and the software development team's quest for efficiency and optimal performance.

Final Thoughts

Let's acknowledge the inevitable: attaining absolute certainty against future renovations is an unrealistic goal. Technologies evolve, past decisions may misalign with the new strategic objectives, and significant migration projects may still be required. Yet, by following diligent preventative measures, you can significantly reduce the risks and be better prepared when challenging decisions become unavoidable.

Before you leave

Decisions regarding code refactoring, rebuilding, or maintaining the status quo can significantly impact your organization's future.

Take that proactive leap and avoid standing at a costly crossroads by subscribing to my newsletter, where I regularly provide valuable insights on building and scaling products. 

Subscribe now

Share this article with others. By doing this, I’ll collect more questions from readers and ensure more relevant articles.

Share

Thank you,

Marina

The common misconception that MVP equals low quality

A Minimum Viable Product (MVP) is a crucial concept to build a successful new product. However, there is a common misconception about this term. MVP is often seen as a product with basic set of features and with low-quality. Therefore, companies consider that they need to choose between building an MVP or a high-quality product.

Unfortunately, this misconception makes many companies (both startups and established brands) averse to the MVP approach. They avoid going the MVP route because they don't want to damage the quality of their product or the reputation of their brand.

What is actually quality of a software product?

Before we talk about quality of MVPs, let's first explore what software quality actually means?

Software quality refers to the degree to which a software product meets specified requirements, customer expectations, and industry standards. It encompasses various quality characteristics, such as performance efficiency, reliability, usability, security, maintainability, etc.

This means that software quality is not limited to the absence of defects only, but it is about the overall user's experience with the system, how easy it is to use, and how it performs in different environments and situations.

For example, a software application with high number of defects is considered to be of low reliability. However, reliability is just one aspect of software quality. Even if an application is completely defect-free, it may still have other issues, such as slow loading times, which indicate poor performance efficiency. Similarly, the software may have a high level of complexity that makes it difficult to make changes and maintain, leading to low maintainability.

MVP is NOT about low quality

The MVP approach was introduced in detail by Eric Ries in his book "The Lean Startup". But in the book Eric never explained MVP as a minimum product from quality point of view. He did not talk about cutting corners during product development, leaving defects in the software and launching a software of poor quality.

The idea of MVP is to think "lean" during product development and to challenge yourself on how to deliver high value with minimal effort and how to maximise the amount of learning during this process. (I've explained in my previous post the concept of MVP in detail).

But even if you deliver a "Lean" product, it needs to be valuable for the user, and even more, it needs to stand out from the competition and delight the user. Otherwise why would the user choose for this new product when there are already other alternatives.

Low quality MVP can be harmful

Delivering a low quality product comes at a risk.

First and most important, a user testing a buggy software would unlikely be delighted. The low quality will lead to damaged user experience, and would hinder the user from seeing the real value of the product.

Low quality is also very harmful for the product development team. Although you might benefit from launching the product faster, this is only a short term win, and over time, ignoring software quality means accumulating so called technical debt, i.e., technical issues that should be fixed at some point.

Maintaining a good quality software is like keeping a house well maintained and tidy. This requires fixing small issues right after they occur, keeping it tidy continuously, and establishing a system or a process to help you keep everything tidy. If you just keep things polished from outside and you hide all the damages and clutter, all issues would accumulate over time. At some point you would need to spend so much time and even money to bring it to a desirable level.

How to approach quality in MVP

Understand what are the quality criteria for your early users

When building a Minimum Viable Product (MVP), the importance of software quality depends on the specific context and goals of the product. While a high level of software quality is always desirable, the MVP is designed to test the viability of the product idea with minimal investment of time and resources. Therefore, it may be acceptable to compromise on certain aspects of software quality in order to deliver the MVP quickly and cost-effectively.

However, before you compromise on any quality aspects, you need to make sure that this is not going to be harmful for the user experience. So it's crucial to identify the key quality criteria that your early users expect and focus on those.

Which are the quality criteria for your product? This can be very different and depends on the type of your business and the expectations of your early users. Evaluate all the quality criteria and consider how much each of them is important for your product at this stage? Questions to consider include:

• What do your users value most?

• How tech-savvy are your early users?

• Is there any critical data your system is processing?

• What is the business impact of a system failure?

• Would a great UX design be the key differentiator for your application?

For example, if you are building a software application for online-shopping, and you discover that your competitor's disadvantage is poor performance, performance becomes a high priority for your product.

Good UX design is typically an important criterion for most users today. However, if for example you are building a highly innovative software product for medical professionals, users might value more the accuracy of the tech algorithm than the UX design.

Define measurable quality requirements

Once you have your key quality criteria, it's a good practice to define concrete measurable quality requirements. These requirements should then be treated in the same manner as the functional requirements (the features), which means that with every release of the software, the product development team should ensure that these requirements are met.

For example, if performance efficiency is you key criteria, a requirement might be defined as "every page should respond in maximum one second".

If Usability is a key differentiator of your product, you might consider a measurable requirement like "the user should be able to purchase the desired product in no more than 3 steps".

When these characteristics are measurable, you can also monitor them and improve over time.

Select the right technology and architecture

The key quality criteria that you've selected drive many decisions in the product development. And one of the key decisions is the selection of the technology and the architecture foundation.

If you have chosen a short-term technical solution (such as a low-code platform that you plan to use until you have a proven business model and you plan to rebuild before scale), the decision you make now would not have a significant impact.

However, if the software you build now will serve as a foundation for the long-term, selecting the right technology and architecture is critical. This includes selecting a foundation framework (potentially a commercial off-the-shelf software), or choosing a frontend, backend, database technology, or specific payment technology. Making the wrong decision could result in the need to fully rebuild the software in the future, which is definitely something you'd want to avoid.

Integrate quality best practices in the development process

An important practice is defining quality coding guidelines, particularly when working with multiple software engineers. These guidelines help ensure that everyone on the team is working towards the same level of software quality.

In addition, integrating automated testing is another important practice that can greatly increase the reliability of the software while also improving the efficiency of testing. Automated testing helps catch defects earlier in the development cycle, allowing for quicker identification and resolution of issues.

Conclusion

Software quality is as important as is the functionality of your product. Even when you are introducing your product to the early users, quality should not be sacrificed in the pursuit of building an MVP faster.

While you should not invest in the most advanced software quality tools or make the software extremely reliable or scalable, a satisfactory level of quality is needed. This will ensure that the user is able to see the real value of your product without hindrance.

In his book"Designing for emotion", Aaron Walter introduces different characteristics that a product should have. Great products are not only functional, but are also pleasant to work with.

MVP is not about implementing partially features only, and ignoring everything else. Instead, an MVP is about implementing a satisfactory level of everything that matters to the user, including both functionality and quality.

The same story repeats over an over. An entrepreneur has an idea, sets a budget, develops a Minimum Viable Product (MVP), but launching the product leads to a disappointment. There is almost no chance that the investment will be returned.

So why is building a successful product anything but easy? Why 90% of the products built today end up with a failure?

In our recent post, we introduced in detail the concept and the definition of MVP. We introduced MVP as a vehicle that moves the users from the current state to their desired state. The key idea behind the MVP concept is that building the MVP is done in iterations, small steps, each of them continuously validated with the user.

Although the term MVP resonates pretty well intuitively with entrepreneurs, the following mistakes are very common and are the biggest contributor to the 90% of failed products.

Mistake #1. Insufficient understanding of the user

I know this is becoming a cliche, but it is still one of the most common mistakes entrepreneurs make in practice. As described above, an MVP is a vehicle to help the users reach their desired state. Starting directly with prototyping and building that vehicle without knowing the users' current state or their desired state, is like targeting with closed eyes. And this is almost never going to work.

Before starting to build an MVP, conducting market research is necessary. We need to have a good understanding of the user, their problem, emotions and desires. Where are the users now, and where do they want to be, what is their desired state. It’s better not to save time when it comes to talking to users and understanding the market. Understanding them better will significantly accelerate the process to achieve a product market fit.

And importantly, when you talk to users, you don't pitch a solution, you only listen to understand the problem. The solution comes after the problem - always.

Mistake #2. Building a basic product with no competitive advantage

Despite being "minimum", an MVP has to provide value to the user, it needs to delight the user, and even more it needs to differentiate from other similar products. A new product, and a new brand in case of a startup company, is likely competing already with existing products (or even sometimes the status-quo can be the biggest competitor). So, an MVP needs to have something extremely valuable, in order to be able to capture the attention of the user, and make the user choose your product instead of another alternative.

This is probably the most difficult part when building a product. Finding that single component, that strategic competitive advantage, that would make a new product stand out from the rest and capture the attention of the user. This requires strategic mindset and knowledge of the competitors and the users, and of course frequent validation with the users.

Mistake #3. Building a working MVP without any prior validation

Because MVP by definition is a "product", entrepreneurs interpret it as a working product, and they start with validation after the working product is ready. They forget that even during the process of building this minimum version of the working product, there should be a lot of validation.

Validating the idea, validating the marketing message, validating the usability of the product via a prototype are often forgotten. But without these validation steps, the risk is high that we are investing in a product that is not going to be accepted by the user.

Validation starts from the moment you have the idea. Validation is done during the Discovery phase, validation is done during the Design when you build an MVP concept or a prototype, and it is done after the product is live. It’s just that the validation techniques you can apply in each stage are different. In the Discovery phase, you rely heavily on interviews or surveys, while once the product is live, you can involve more advanced quantitative techniques, like product analytics or A/B testing.

Mistake #4. Failing to keep it "Minimum"

The meaning of "Minimum" in the term MVP is often well understood, but still applying it in practice is somehow difficult. Minimum viable products often become products with a lot of features that are not necessary in the first iterations. The result is not only spending a lot of time and money on development, but also making a product that is complex and confusing for the user.

Building a minimum viable product requires a lean mindset. It requires critical evaluation of the value and effort of the features you want to implement, and applying the 80/20 rule - selecting the key features that bring the highest value to the user.

Mistake #5. Forgetting the quality of the product

Focusing on minimum features only, and forgetting to embed certain level of quality when building the product is a common mistake many entrepreneurs make. But insufficient usability, reliability, or performance efficiency for example, will damage the user experience of the user and will hinder the user to see the real value of the product.

Even when building an MVP, quality is important. You don't need to invest in the most advanced quality best practices, but you should aim for sufficient quality that is necessary to provide good user experience.

Mistake #6. Committing to output-driven roadmaps

The term roadmap is often understood as a timeline with milestones, each with concrete features to be delivered. But this way of working now belongs to the traditional product development.

When we build a minimum viable product, of course we need to select the scope of the product, we should select which user stories should be implemented. But we don't commit to a long-term roadmap with concrete milestones. After the product is live, we should focus on learning from customers and shaping the next iterations of the product based on the customers feedback.

And importantly, within this process, the focus should not be on the Output, but on the Outcome. So when setting the milestones, it's better to set an outcome-oriented milestone, what value do you want to bring. For example, "in this quarter we want to decrease the delivery time by 25%", is better than "in this quarter we want to build these features".

Mistake #7. Aiming to grow before having a "sticky" product

In their book Lean Analytics, Alistair Croll and Benjamin Yoskovitz propose that every successful startup company goes through five stages and in this order: Empathy, Stickiness, Virality, Revenue, and Scale.

The first phase is Empathy, where the focus is to learn your users and their needs. Then you move to Stickiness, the phase when you are building the product. The goal at this stage is to build a product that resonates with the users and is very well accepted. Only after you optimise the product, and you make it "sticky" for the user, you should move to the next stage, where the focus is growth.

And this is where it often goes wrong in practice. Too often, this order is not respected and the focus is on Virality and increasing user acquisition as much as possible without understanding what the users are going to do with the product.

When improving your product, focus on one objective at a time. What to focus on depends on your business, but as a general rule, first stickiness, and then growth.

The MVP concept still works

Even after two decades since the term MVP was introduced, the theory is still the same. The MVP concept still works. And there are great examples of extraordinary products that all started as MVPs, for example, Dropbox, Facebook, Instagram. It is all about finding how this basic MVP concept could best be applied for your specific situation and the product you are building.

Minimum viable product (MVP) is one of the most important concepts in the process of building a software product. The concept was introduced by Frank Robinson in 2001. However, even after more than two decades since the term was introduced, and more than a decade that this concept is actively practiced in the tech world, the statistical data is still disappointing. About 90% of the startups end up with a failure.

While the term Minimum Viable Product resonates quite well intuitively with everyone, there is still a lot of confusion about what this term actually is, and even more how to apply this concept in practice.

Eric Ries, the author of the bestselling book ,Lean Startup, describes MVP as a ,version of a new product that allows a team to collect the maximum amount of validated learning about customers with the least effort.

But is MVP actually a product? This is often a debate. Some people believe that an MVP is only a prototype, it should never be a working product. For others an MVP is the minimal version of a live working product that we can sell to customers.

Before we talk about the definition, let's first understand the idea behind the MVP concept.

To build a product, we need to have users of that product. A user should have a need, or a desire, for example a problem to solve or a desire to become better in something. We build a product in order to satisfy this need of the users. A product in simple words is a vehicle that can move the users from the current state to their desired state.

But the idea we have for a product relies on a lot of hypotheses. This is why we don't build the complete product in one go. We build it in iterations, as small as possible loops, and within each iteration, we do validation with users. During this process, we validate our hypotheses, starting with the riskiest one. The goal of this approach is failing fast: If the initial idea is not valid, it's better to fail fast before making a big investment. And this is the meaning of the MVP concept - working in small iterations, and frequent validation with users.

To conduct this validation, we need something tangible, something we can show to the user and get feedback, we need an MVP.

So is the MVP a working product? Not necessarily. Depending on what we want to validate, we can use different types of MVP. A landing page is an example of a marketing MVP type, where we test the marketing message and the idea we have for a product. A wireframe or a mockup MVP can be used to test the user experience of the product. Wizard of Oz is another type of MVP, it is a working product, but where some of the basic but complex functionality is replaced by manual work in the background. So with Wizard of Oz MVP, we validate only part of the functionality of the product.

And because implementing the product is an iterative process, in practice we would typically go through multiple stages. An example scenario would be to start with a landing page MVP, then a wireframe MVP, a prototype MVP, maybe move to a Wizard of Oz MVP, and at some point during this journey, we would have the minimum viable version of a live product that we can sell to users. Having a live version of the product does not mean we stop there, instead we should iterate, improve, add more features and gradually increase the value to the users.

With every iteration during this process, the feedback we get from users allows for changing direction, correcting the course and getting closer to the product market fit.

Let's now go back to the definition of MVP. The "P" in MVP could refer to a product, but also a prototype, or concept of the product. Sometimes to avoid confusion, the MVP versions that are not working products are referred to as "MVP tests" or even "Minimum viable prototypes", while MVP is reserved for the live working sellable product.

I would not strongly advocate which terminology is best to use. Choose the one that you best feel comfortable with. But no matter what term you use, don't forget to apply frequent validation, not only after the working product is ready, but during the whole journey, starting from having the idea and onwards.

Until when do we use the term MVP for a new product, so when is it not "Minimum" anymore? I don''t think there is a clear boundary between MVP and a product. I'd say that's the moment when we feel confident about the product, when it resonates well with the market, and when we have found the product market fit.

Wasteful process can lead to enormous losses

Waste is any expense or effort expended that does not add value to the customer.

Waste is unwanted and unusable, and when not managed well, can lead to enormous losses. In the fashion industry for example, it is estimated that better waste management could save €4 billion per year up to 2030. Around 35% of waste occurs only during the supply chain before the garment reaches the end consumer.

I am not aware of such estimates in the domain of software product development. As this domain is a lot more abstract compared to manufacturing, quantifying waste is a far more difficult topic. I believe however that in software development, it is even more likely to have waste, because it is often hidden and not obvious to identify. Organisations who are aware of the importance of eliminating wasteful practices certainly benefit from higher productivity and more satisfied customers.

Common types of waste in software development

The term waste is closely related to the concept of lean manufacturing, a methodology that focuses on minimising waste and maximising productivity within manufacturing systems. The origins of lean manufacturing are based on the Toyota Production system, developed in the 1950s by Taichi Ohno, a Japanese industrial engineer. Taichi devised the seven wastes in a manufacturing system: Inventory, Overproduction, Extra Processing, Transportation, Waiting, Motion and Defects.

Lean as a philosophy has emerged over the recent decades and became an inspiration for all branches of industries. The idea of removing waste became so popular in the 90s that inspired lean software development as well as Kanban. The seven types of waste in lean manufacturing have been translated to seven types of waste in software development.

Although this translation captures quite well some common inefficiencies in software development, we should not fully rely on it, because software development is a very different domain from manufacturing. In manufacturing a physical product is built following a defined number of steps, while in software development we built something intangible following a very complex process. In manufacturing building a second product means repeating the same process, while in software development, the source code can simply be reused without the need to reimplement. There are types of wastes in software development that are not typical for manufacturing and vice versa.

In the table below I have listed the most common inefficiencies I have encountered in many organisations. The first seven of them originate from lean manufacturing, the others are common wastes but specific for software development.

How to remove waste in software development

Sometimes organisations integrate focused waste reduction approach. Employees dedicate a special period when they work on waste removal. This can be a useful practice because this is time when there are no other feature development priorities. However, as the time dedicated to reduce waste is usually limited, this period often results in cleaning the issues that are very easy to tackle, e.g. removing unused code, cleaning the backlog. Problems that require more crucial change, like redesigning a system, or adjusting the governance approach would remain unsolved.

Another practice is continuous waste reduction, when part of the product development (10–15%) is continuously allocated on improvements. The positive side of this approach is that it is a continuous process and it does not block the feature development. In practice however, it can happen that this 10–15% never get a priority due to pressing deadlines for delivering features. To make it happen, teams should have a good discipline and make sure to regularly include waste reduction issues in their sprint backlog.

While both approaches are valuable, what I often see missing is that they capture limited scope. In the first approach, the decision of which problem to tackle might be made by an individual, in the second approach, it is typically a team’s decision. During a retrospective meeting, the development team might identify a critical bottleneck in their work, but if this is not something the team could solve autonomously (e.g ineffective communication with another team or with the product managers), they would leave the problem unsolved.

And that’s unfortunate, because it might lead to spending effort on solving small issues while the basis is still wrong. The product development is a complex system and requires a joint effort between different stakeholders and teams. There should be an efficient flow of information in the complete system to make the work of this system optimal. One way to address this challenge is to form a team of diverse roles, who will make decisions about waste reduction in a more strategic and structured way.

Identify the wastes in your process

A good way to start with waste reduction is to evaluate each phase of the feature development process we follow. We evaluate whether each phase is performed effectively and efficiently and whether it produces high quality as output. The list of common wastes (Figure 1) can be used as a guidance, but we should not limit our thinking to only those types of waste. Each process is unique and might very well contain its own unique hidden wastes.

For example, the product development might follow the process shown in the Figure 2.

When we are evaluating the Development phase, we might for example discover a high percentage of user stories that are in “In Progress” state. This is one of the common wastes listed above — Partially done work.

The team should discuss and identify the reason behind this problem. It can be due to frequently shifting business priorities, lack of domain knowledge in the team, or simply inefficient backlog management. Although this issue might be discovered by the software engineers, the waste is caused in the previous phase, during feature requests prioritisation, and should be addressed by the product owners/managers.

Another example might be a high number of defects discovered in the Acceptance testing phase. This is a sign that there is insufficient testing in the previous stages. By observing the types of defects identified in Acceptance testing, we could discover that there is insufficient unit testing or integration testing. Once the waste is identified, the solution becomes obvious — e.g. adjusting the test strategy and increasing the code coverage threshold for unit tests.

Prioritise

Next step is prioritisation. This is an important step and should not be avoided. The whole process of waste reduction causes effort, so this effort should be spent in the most optimal way to get the best outcome.

We could use a simple prioritisation technique, as shown in Figure 3, where each waste identified is assigned Impact value and Effort value (the effort required to eliminate the waste). Once we have the wastes plotted in this matrix, we prioritise, by choosing those that are with high value and reasonably low effort. We could for example select the top three types of waste to be solved and integrate them in the backlog.

Of course, making an accurate quantified estimation is usually unfeasible, but it is also impractical and unnecessary. It’s sufficient to prioritise wastes based on our best guesses.

Removing waste is a continuous and gradual process. People have the natural tendency to accommodate easily to wasteful practices, so setting up a continuous process that integrates lean thinking is important.

Final thoughts

Waste usually happens due to making mistakes or unawareness. But as long as we are learning from these mistakes, this is not really a waste. It becomes a waste if we stay ignorant to these mistakes and we don’t do anything to make an improvement.

This post in part of the series “Successful digital product growth”. Check bellow the remaining articles:

#1 Is your digital product ready to grow?

#2 What is an effective product strategy?

$3 Set up a product development team for success

#4 How to remove waste in software development

#5 Unlocking the architectural scalability of software products