It really depends on where you are coming from. There is a good argument for "never refactor" if you just look at it on a time cost vs new feature benefit point of view.
The common arguments in favour of refactoring tend to be more abstract
Maintainability. If the code is constantly worked on and "up to date" then its going to quicker to add features/fix bugs
Skills up to date. If you keep refactoring to the latest framework, programming language etc then your devs are always working with cutting edge tech, are easier to hire