There's this fact/urban legend that if you put a frog in hot water, it will jump out. But if you put a frog in cold water, it will stay in. If you then heat up the water, the frog does not notice and eventually gets boiled to death. This is most likely not true, but it serves as a fantastic analogy here.
Technical debt often behaves like this. If you introduce a developer to a codebase that's already steeped in technical debt, they will balk and either not want to take on the project or start it from the ground up. But any project, even ones that start cleanly, over time accrue technical debt, often stemming from small decisions that initially don't appear to have long term ramifications. This may even be statistically correct, but over time at least some of these decision have a bigger impact than expected and they become the precipitation points on which technical debt issues start to form.
My question is: what conditions are sufficient for successful refactoring?
To continue the analogy, you're asking how the frog can make sure it doesn't get boiled to death once it's in the (now cool) pot.
First of all, it requires us to define what we consider to be too hot, and that's already a significant can of worms. So let's sidestep that discussion and conclude that we have some kind of metric in place, whether it's a specific temperature, a quick rise in temperature, ...
Next up, what good does that metric do if we don't give the frog the right tool to track it (thermometer, timer, ...). This means that you need to monitor the situation, such as:
- The velocity of the team over time. As the technical debt increases, you'll see that the devs need more and more time to deliver the same amount of features.
- Static analysis of the codebase, which can indicate a growing list of concerns/warnings.
- Listening to your developers. Often, technical debt is observed and communicated well before actual action is taken.
Okay, now we have a frog who knows the pot is heating up. But did we make sure that the frog jumped in a pot that it's actually capable of getting out of again?
The equivalent part to that analogy is having reliable (up to date) documentation of the business requirements and a testing suite (which ideally should be the code-equivalent of the requirements documentation). This enables you to refactor any part of your codebase while maintaining confidence that the refactoring isn't changing the actual behavior.
Really, all I've done here in this answer is list the main ingredients to create a dev team that does not let technical debt pile up:
- Agree on the definition of technical debt and good/bad practice
- Observe the current state of the technical debt and good/bad practice
- Prepare to have to fix the current state of the technical debt and good/bad practice
I can't spell the whole process out, because this is way too contextual for every company and situation. If there were a one-size-fits-all solution to this, it would already be a widely distributed guideline. As it currently stands, technical debt requires a balancing of business and technical needs and requires cooperation with the non-technical side of the company who are usually in control of the budgets and requirements, and that's not something you can prescribe in a guideline.