The treatment of the three men may not appear all that drastic. But there is a message here about the way the IOC is prepared to react to the demands that the administrators of world sport and, in particular, football must become more accountable and transparent.
It would be too much to say that this is sport's equivalent of the Arab spring. However, it shows that the leaders of the Olympic Movement now seem prepared to demonstrate that international sport bodies are no longer cosy clubs where whatever happens inside the club remains hidden and does not concern outsiders.
Before you think I am reading too much into all this just consider what has happened.
In November last year, the BBC Panorama programme made a series of allegations about the three men. This was not the first time the media had made allegations about leading sports administrators and their alleged financial misdeeds. The seriousness of the allegations against Havelange, Hayatou and Diack could not be doubted.
Panorama alleged that Havelange, who combined his FIFA Presidency with membership of the IOC, had received $1 million (£639,000/€746,000) from ISL, one of the most important of the many companies that FIFA dealt with under the Brazilian's leadership. ISL owned World Cup television rights and its collapse back in 2001 with debts of $300 million (£192 million/€224 million) has been at the heart of the many corruption allegations that have swirled around FIFA in the last decade. Havelange denied all allegations and to date there has been no action by football's governing body.
However, even before the IOC Ethics Commission had reported, Havelange, who seemed to be a permanent IOC member having been on the committee for 48 years, suddenly resigned. He is believed to have cited health reasons – he is after all 95. Yet the outside observer could not help but note that his health problems emerged just as the Ethics Commission was due to report. Stories floated around that he even faced expulsion from the IOC.
With Havelange a "private person", as the IOC President, Jacques Rogge, put it, the Committee was left to report on Hayatou (pictured left) and Diack (right). The sentences they have received are light but their consequences go beyond the actual punishment to what they say about the running of world sport.
Diack, who as President of the International Association of Athletics Federations (IAAF) is an IOC member, was warned for receiving a series of payments totalling 52,680 Swiss francs from ISL.
But it is how the IOC dealt with Hayatou that is very significant. The man, who since 1987 has been President of the Confederation of African Football (CAF), was alleged to have received French francs 100,000 from ISL. The money was paid in 1995 and the Cameroonian admitted to IOC investigators that he had received payment in cash. His justification: "the state of banking technology in some countries, making cash payments was current practice at the time". He also insisted, as he has done since the BBC broadcast, that the money was necessary to finance CAF's 40th anniversary dinner.
Hayatou's reprimand was based on the fact that the IOC Ethics Commission found the documents had been drawn up "a long time after receipt of the funds" and could not "guarantee that these payments were indeed made into the CAF accounts". For Rogge, this merited Hayatou getting, as he put it, a "yellow card".
Observe that the event happened long before he became a member of the IOC. But this is where the words of the Ethics Commission acquire tremendous significance: "The Commission notes that, although the acts took place at a time when Mr Issa Hayatou was not yet an IOC member, he was the then vice-president of an international federation, FIFA, a constituent of the Olympic Movement and that, as such, he was obliged to respect the fundamental principles of Olympism."
The words of Rogge go even further, "We will not hesitate to act. The wider world will acknowledge that the IOC means business and is accountable and transparent."
Rogge's words suggest that the IOC now sees itself as a sort of supervisory sports body which can investigate actions even when they do not directly relate to the Olympic Games. Should the IOC follow up on this new principle, then it could have far reaching consequences for world sport. This will not only impact on the IOC's relations with other sports bodies but it could lead to an unravelling of the remarkable organisation that Antonio Samaranch created as the linchpin of world sport.
Samaranch wanted to bring all the world sports bodies under the IOC's umbrella and made many of those who ran the various sports federations members of the IOC. It did not matter how these federations were governed as long as their sports men and women competed in the Olympic Games. Indeed, he saw their IOC membership as tying them to the Olympics.
Rogge, on taking over the IOC Pesidency, had lauded Samaranch as his sports political guru saying he had learnt about sports politics from the Spaniard. But with sports organisations under increasing scrutiny and demands that they follow good governance, he is now saying that the IOC will lay down the law on how these sports bodies must be governed. Not even football, the most powerful of these bodies, is immune.
This is, of course, very welcome, more so in a world where sports organisations like FIFA have until now shown little awareness of acceptable governance and accountability. Rogge is setting out to create a new IOC and a new world sports order. How he works this out will demonstrate whether sport can reform itself without legislative guidance. And, as he sets out on this mission, Fifa, and indeed all major international sport organisations, need to pay close attention. It may be an uncomfortable road but it is a road world sport, particularly football, must travel to regain public trust.
Mihir Bose is one of the world's most astute observers on politics in sport and, particularly, football. He formerly wrote for The Sunday Times and The Daily Telegraph, and was the BBC's head sports editor. Follow Mihir on twitter.