14 October 2021
Running at a fixed lag behind major technology selection choices to mitigate risk.
Early adopters take on a lot of risk, as do stragglers. There's a sweet-spot around 6 months where most problems are ironed out (or at least known) while it's still new enough to be supported.
When dealing with key project dependencies of a software platform, this risk can have a profound effect on end users. Common examples include language selection, upgrading major versions of libraries and 3rd party service suppliers. In all these cases there are similar risk factors that can affect developer productivity and platform stability. Ultimately this affects the end user experience through the efficacy of the team at fixing bugs and making changes.
Common issues faced by early adopters:
- A lot of bugs (it's not had time to be battle hardened by broad user exposure).
- Dependency incompatibility (breaking change not had time to work its way through the ecosystem).
- Absence of documented problems or workarounds (the community hasn't had time to hit these issues, report them and devs to fix).
- Solution divergence - you have to "pick a side" on choices that may transpire to be dead-ends, causing rework.
- Solution obsolescence - quick hacks to work around issues are replaced by "correct" solutions, causing rework.
- Icebergs - seemingly small decisions that can have big unforeseen consequences.
- Critical bugs that are "Fixed in next release" inevitably leading to using Snapshot dependencies and a cycle of heavy bugs and fixes (or worse, getting stuck on the Snapshot).
What the end user sees; Things keep breaking, and progress is periodically slowed.
Common issues when you are lagging too far behind:
- Incompatibility of modern libraries/tools/technologies limits what features can be added.
- Inability to upgrade due to "dependency deadlocks" (can't upgrade X because of Y, can't upgrade Y because of Z, etc) - this can have several causes.
- Bugs not getting fixed (instead, increasing numbers of work arounds).
- Difficulty getting developers familiar with- or willing to work on- the platform.
- General difficulty making changes to the project.
What the end user sees; Some features are “not possible”, the product is inflexible.
How we deal with it
- Lag behind the release schedule by a fix time - I recommend between 3 and 6 months.
- Always use LTS versions where available.
- Run a periodic tech state review - where are the risks? what are we behind on?
- Schedule Major upgrades as high risk items with low urgency.
- Treat working legacy systems as high risk items - modernize > migrate > mitigate.
For libraries in particular, I'd advise greedily taking "bug fix" releases (considered in sprint/ad-hoc), periodically taking "minor" releases (considered in sprint/every few weeks) and occasionally taking "major" releases (considered per 1 or 2 months).
Alternatives
There are other ways that work better for large teams to deal with the same risks. To name a few:
- Fix forward (if your team is large enough and your breakage/risk tolerance is high).
- Obsolescence architecture - delete old components and replace them with functionally equivalent new components (if the cost of upgrade is too high and a “bang/coast/bang” expense profile is preferable).
- Microservice architecture (risk containment, rollback policies).
Whichever approach you take, I advise making a purposeful choice and applying it consistently.
