International Olympic Committee (IOC) Coordination Commission chair John Coates has warned Tokyo 2020 that their current budget cap of $20 billion (£16 billion/€18.8 billion) risks "giving a bad impression" to cities considering bidding for the Games.
The Australian was speaking at the plenary session of a first official inspection by the Coordination Commission in almost 17 months here today.
He emphasised to Japanese organisers how they "continue to be pleased" with their performances.
But he then criticised the figure announced this week by Tokyo 2020 President Yoshirō Mori as a maximum ceiling rather than an actual budget.
"We haven’t agreed to the $20 billion budget figure," said Coates, also one of four IOC vice-presidents.
"We think there are significant savings to be made.
"It is important we do make those savings and we identify them now - because we don’t want to do this later.
"We have future candidate cities for the Olympic Games looking here.
"We don’t want to give the wrong impression as to what it costs to host the Olympic Games."
Tokyo 2020 are aiming to release the first version of their operational and infrastructural budgets by the end of 2016.
It comes after a difficult period starting in late September when a Metropolitan Government Task Force warned that drastic changes must be taken to avoid the budget ballooning to ¥3 trillion (£22 billion/$30 billion/€26 billion).
This figure was more than four times the initial one given during the bidding phase.
One challenge for Tokyo now involves defining the breakdown in contributions between the differing levels of Government.
The Organising Committee's portion of the budget is expected to come to ¥500 billion (£3.55 billion/$4.7 billion/€4.2 billion), but a further breakdown between municipal and central authorities is not yet confirmed.
Concerns also remain over whether overlay costs at existing and temporary venues is grouped within the Organising Committee or infrastructural costs.
A reluctance to have to further increase the budget once it is announced is another factor thought to be delaying the announcement - as well as a possible explanation as to why Tokyo are reluctant to comply with Coates' demands.
Financial fears were a major reason why Boston and Hamburg bids for the 2024 Summer Olympics and Paralympics proved unsuccessful, as well as for a Rome attempt being "self-suspended" in October after Mayoral opposition.
A key facet of the IOC's Agenda 2020 reform process passed in 2014 was to reduce the cost of bidding but - despite IOC President Thomas Bach's claim that it has directly contributed to the Budapest, Los Angeles and Paris bids for 2024 being launched - fears over the rising cost of the Games remain high.
As Coates mooted, further budget rises for Tokyo would add to this feeling, especially because the Games has supposedly benefited from Agenda 2020-inspired cost cutting measures.
Coates emphasised today how these venue charges introduced last year supposedly trimmed $1.8 billion (£1.4 billion/€1.7 billion) off the budget.
He also predicted that further alterations to venues for swimming and rowing and canoe sprint could save an additional $400 million (£319 million/€377 million), "at a minimum".
Tokyo 2020 President Yoshirō Mori also spoke at today's plenary session.
He apologised to Nigeria's Coordination Commission member Habu Gumel, who is also part of the International Volleyball Federation Board of Administration, for their failure to so far confirm a venue for indoor volleyball.
A decision between a new Ariake Arena in Tokyo Bay and an existing venue in Yokohama is due to be made by Christmas.
Venue and budgetary-related matters are set to among major items on the agenda for the next two days as 13 of the 16 Coordination Commission members attend the meeting.
Burundi's Lydia Nsekera, France's Guy Drut and Britain's International Association of Athletics Federations President Sebastian Coe are all absent.
Italy's Association of Summer Olympic International Federations President Francesco Ricci Bitti is attending his first meeting since replacing International Judo Federation and former SportAccord President Marius Vizer on the panel.