Many Tottenham Hotspur fans are reminded of the wisdom of this as the club, having summarily replaced one of football's most venerable and successful managers with a younger man, who failed at rivals Chelsea, make a disappointingly inauspicious start to the season.
There is a sound argument for believing that in sport as in life, things are usually best left alone.
A similar thought must be running through the minds of those employed by the British Olympic Association (BOA) at the moment.
As the gold dust settles on arguably the greatest-ever Olympic Games, in which BOA staff played such a pivotal and supportive role, the air is thick with speculation of changes and upheaval following chairman Colin Moynihan's impromptu decision to step down before completing his second term of office. This took everyone by surprise, not least Boris Johnson.
In fact, when the good Lord summoned a few of us to have lunch with him his favourite Asian eatery in Westminster to impart the news, by chance the London Mayor, his bike parked outside, was already tucking into his King Prawn Piri Piri at a nearby table. He almost fell off his chair when Moynihan took him to one side to tell him he was stepping down.
Boris was still shaking his head as he put on his bicycle clips and helmet before pedalling back to City Hall.
I am told that Seb Coe was similarly taken aback when Moynihan informed him he was getting on his own bike. But since then there has been mounting speculation that the London 2012 chief himself is now in pole position to take over from his fellow Tory peer when the BOA hold their annual meeting in November.
Moreover, it is rumoured that should this happen Coe will be making changes and bringing in some of his own London 2012 cohorts to run a revamped BOA.
All of which must be deeply unsettling for those in the current team which performed so ably in the build-up and during the Games, particularly in orchestrating the facilities for the athletes and their families at Team GB House.
From a personal perspective I thought the BOA's media operation ranked among the most professional of any global sporting event I have every covered.
So what is about to happen post-Moynihan? Clearly, the Paralympics are set to follow the Olympics as another glorious triumph for Coe. Because of them, he has become the most sought-after figure in sport with the BOA chair an obvious post-Games job opportunity for the overlord of the rings.
Initially he had no thoughts of taking over from Moynihan but influential voices in Whitehall and at Westminster are now urging him to stand, believing he is the ideal figure to take the organisation forward to Rio in 2016.
But I understand Coe will not tout for the role or contest an election. He would have to be installed by unanimous choice, and this may prove difficult with other candidates including Moynihan's close friend, the award-winning Sussex businessman Richard Leman, an Olympic hockey gold medallist now President of GB Hockey, already in the frame.
Sport's oarsome knights Steve Redgrave and Matthew Pinsent and British Rowing chair Di Ellis, along with current BOA vice-chairs David Hemery and Albert Woods normally would also be in contention, as might Coe's number two at London 2012 Sir Keith Mills (although it is thought he is earmarked either to chair a new combined UK Sport/Sport England or the Premier League).
It also has to be said that there are those on the National Olympic Committee who would feel more comfortable with someone from within their own ranks to head up the organisation.
However, so high is Coe's stock after the 2012 Games that such an illustrious appointment will be hard to resist, and from Coe's viewpoint it could only enhance his campaign to become President of the International Association of Athletics Federations (IAAF) in 2015.
Should Coe take the chair he will need to drastically overhaul the cash-strapped BOA's finances – its 2012 operation was impressive but costly, worsening an anticipated seven-figure deficit – and, it is said, supervise a staffing shake-up with the future of number of highly-paid key figures under review, including director of performance Sir Clive Woodward, chief executive Andy Hunt and commercial director Hugh Chambers, all Moynihan appointees.
It would not surprise me if Woodward elected to move on to sporting pastures new. England's World Cup-winning rugby coach was Deputy Chef de Mission at both the Beijing 2008 and London 2012 Games but was overshadowed by the omnipresent Hunt, who, according to the Daily Telegraph, "at times seemed more like a cheerleader than Chef de Mission."
This may be somewhat harsh but I do feel Hunt should have been the midfield anchorman and allowed the more experienced sports-savvy Woodward to have the field-marshal role up front.
Without Moynihan's backing, Woodward may feel it is time his proven mentoring talents are given a fresh challenge.
One executive definitely departing is American Darryl Seibel, their top-notch director of communications who is returning to the United States in October. He has a most capable aide in Miriam Wilkens but if he is to be replaced there is a ready-made candidate in Jackie Brock-Doyle, Coe's own communications chief at London 2012, where sports director Debbie Jevans would also be in line for a move to the BOA should either Woodward or Hunt depart.
All of which is hugely speculative, as is Coe's own future. The BOA chair is an unpaid post, and although a reasonably wealthy man, Coe will tell you he still needs to earn a living.
This he can do to substantially by resuming his earlier role as an ambassador for Nike, and on the corporate speaking circuit with so many absorbing London 2012 after-dinner tales to relate he will be in high demand to trouser up to £25,000 ($40,000/€32,000) a time, at least putting his income on a par with Premiership footballers.
He also has a part-time occupation as David Cameron's new "sports legacy ambassador", although I doubt if even he is sure exactly what that entails.
One chair which he is now unlikely to seek is that of the British Boxing Board of Control (BBBofC), where a number of disaffected of licence holders, led by top promoter Frank Warren, are attempting to oust the under-fire incumbent Charles Giles.
It is now likely they switch to wooing the now available Moynihan, who, like Coe, is a former Board steward who loves the fight game.
Whatever the future holds for sport's respective lordships, it appears the gloves are off in both the boxing and Olympic rings. Seconds out!
Alan Hubbard is an award-winning sports columnist for The Independent on Sunday, and a former sports editor of The Observer. He has covered a total of 16 Summer and Winter Olympics, 10 Commonwealth Games, several football World Cups and world title fights from Atlanta to Zaire.