Sometimes, we inherit broken CI/CD pipelines and are handed the task of fixing them.
At this point, we have two choices:
- Declare it a disaster zone. Decide that the existing pipeline is beyond salvation and rewrite it from scratch (because obviously, you can do better).
- 2. Take a closer look. Realize that only a small section is causing issues, and a targeted fix might be enough. But then comes the risk—if you patch it, and it still performs terribly, people will forever associate the broken pipeline with you!
Now, beyond personal pride, we must also consider the cost and who’s paying for it.
In an ideal world, we’d always opt for a full rebuild—a beautifully structured, clean, and efficient pipeline that we could admire like a masterpiece. But in the real world, the business (the budget holder) might be on the clock, needing a production release in a few hours.
A Real-World Analogy: My Shed Roof
Recently, I faced a similar dilemma—not with code, but with my garden shed.
Last month, during Storm Éowyn, the felt roof took a beating, and a large section was torn off. When I first saw the damage, my instinct was:
“I need to replace the entire roof!”
Today, I finally got around to fixing it. But this time, I was the budget holder—and my budget wasn’t money, it was time.
I had multiple garden jobs to tackle, and if I spent hours replacing the whole roof, I wouldn’t get around to them until next weekend.
So, I took a more pragmatic approach.
On closer inspection, I realized the damage was only at the front edge. Because of the overlapping nature of the felt, I could replace just one strip at the front, rather than tearing off the entire roof.
Is it perfect? No.
Will it last as long as a full replacement? Probably.
Does it matter? Not really.
Why? Because the repair was driven by my time budget. (with my extra time today I fixed the alignment on the garden pub door so they where no longer sticking – great result)
What Does This Have to Do with DevOps?
When you’re fixing someone else’s code, always discuss possible solutions with the budget holder/product manager/agile delivery manager
- Highlight that patching the issue might introduce some technical debt.
- But also consider when that debt actually needs to be repaid.
Many articles make technical debt sound like a major concern—and for long-lived products, it certainly is. But in many situations, how many products truly last over two years?
From my experience, very few projects are actively developed for two years or more. Unless you’re working on a fast-evolving product with a long-term roadmap, short-term fixes—like my shed roof patch—are often a more budget-conscious approach than shouting “technical debt!” and pushing for a full rewrite.
Final Thought
Before deciding between fixing or rewriting, consider:
- How long will this product be around?
- What’s the urgency?
- What’s the budget (time or money)?
Sometimes, a well-placed patch is all you need to keep things running—just like my shed roof.
